[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:06 dist-test-slave-cm7b krb5kdc[1209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:06 dist-test-slave-cm7b krb5kdc[1209](info): set up 2 sockets
Sep 24 00:14:06 dist-test-slave-cm7b krb5kdc[1209](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 00:14:07.311626 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:07.430119 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46749
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:07.512058 1222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:07.512260 1222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:07.512290 1222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:07.513439 1222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:07.513482 1222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:07.513496 1222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:07.513509 1222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:07.513561 1222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:07.515076 1222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:46749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:07.515370 1222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:07.515630 1222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:07.520592 1228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:07.520627 1230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:07.520644 1227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:07.520677 1222 server_base.cc:1034] running on GCE node
I20260924 00:14:07.520980 1222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:07.521301 1222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:07.522429 1222 hybrid_clock.cc:648] HybridClock initialized: now 1790208847522407 us; error 49 us; skew 500 ppm
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:07.525022 1222 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:07.525406 1222 webserver.cc:460] Webserver started at http://127.1.44.190:39613/ using document root <none> and password file <none>
I20260924 00:14:07.525599 1222 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:07.525640 1222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:07.525728 1222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:07.526746 1222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "b17062a053224a9ca96728c0590e3270"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "a9fbb66e3fc6b0546067b2f9b9675491"
server_key_iv: "1d75a8c99756d0b05f5fca5a9e69b192"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.527064 1222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "b17062a053224a9ca96728c0590e3270"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "a9fbb66e3fc6b0546067b2f9b9675491"
server_key_iv: "1d75a8c99756d0b05f5fca5a9e69b192"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.528446 1222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:07.529136 1237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.529304 1222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:07.529394 1222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "b17062a053224a9ca96728c0590e3270"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "a9fbb66e3fc6b0546067b2f9b9675491"
server_key_iv: "1d75a8c99756d0b05f5fca5a9e69b192"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.529448 1222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:07.545764 1222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:07.547955 1222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:07.548082 1222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:07.551863 1222 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:46749
I20260924 00:14:07.551918 1289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:46749 every 8 connection(s)
I20260924 00:14:07.552237 1222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:07.553200 1290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:07.554872 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1222
I20260924 00:14:07.554986 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:07.555789 1202 external_mini_cluster.cc:1428] Setting key 83d19c4415ec9a7e4a4d98d3934d7ebb
I20260924 00:14:07.555984 1290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270: Bootstrap starting.
I20260924 00:14:07.556684 1290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:07.557026 1290 log.cc:824] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:07.557763 1290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270: No bootstrap required, opened a new log
I20260924 00:14:07.559437 1290 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17062a053224a9ca96728c0590e3270" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46749 } }
I20260924 00:14:07.559587 1290 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:07.559649 1290 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17062a053224a9ca96728c0590e3270, State: Initialized, Role: FOLLOWER
I20260924 00:14:07.559787 1290 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17062a053224a9ca96728c0590e3270" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46749 } }
I20260924 00:14:07.559885 1290 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:07.559929 1290 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:07.559980 1290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:07.560761 1290 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17062a053224a9ca96728c0590e3270" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46749 } }
I20260924 00:14:07.560884 1290 leader_election.cc:302] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17062a053224a9ca96728c0590e3270; no voters:
I20260924 00:14:07.561079 1290 leader_election.cc:288] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:07.561106 1295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:07.561218 1295 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [term 1 LEADER]: Becoming Leader. State: Replica: b17062a053224a9ca96728c0590e3270, State: Running, Role: LEADER
I20260924 00:14:07.561362 1290 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:07.561369 1295 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17062a053224a9ca96728c0590e3270" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46749 } }
I20260924 00:14:07.561750 1295 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17062a053224a9ca96728c0590e3270. Latest consensus state: current_term: 1 leader_uuid: "b17062a053224a9ca96728c0590e3270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17062a053224a9ca96728c0590e3270" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46749 } } }
I20260924 00:14:07.561836 1295 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:07.562253 1297 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17062a053224a9ca96728c0590e3270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17062a053224a9ca96728c0590e3270" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46749 } } }
I20260924 00:14:07.562350 1297 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:07.562438 1302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:07.563117 1302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:07.564743 1302 catalog_manager.cc:1347] Generated new cluster ID: 9483d9bc4de3452ead71d9527f55122b
I20260924 00:14:07.564800 1302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:07.571167 1302 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:07.571875 1302 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:07.572638 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:07.556688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.1.44.190:46749)
0924 00:14:07.556910 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:07.556919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:07.557154 (+ 235us) server_negotiation.cc:407] Connection header received
0924 00:14:07.559418 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:07.559424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:07.559470 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:07.559605 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:07.561248 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.562947 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.564502 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.565030 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.565170 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:07.568522 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:07.568526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:07.568527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:07.568540 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:07.570680 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:07.571268 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:07.571271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:07.571272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:07.571369 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:07.571606 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:07.571607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:07.571608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:07.571823 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:07.571989 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:07.572275 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 00:14:07.572420 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
I20260924 00:14:07.580511 1302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17062a053224a9ca96728c0590e3270: Generated new TSK 0
I20260924 00:14:07.580710 1302 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:07.610044 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46749
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:07.713064 1316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:07.713279 1316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:07.713312 1316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:07.714471 1316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:07.714514 1316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:07.714540 1316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:07.716025 1316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46749
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:07.716257 1316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:07.716502 1316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:07.722466 1321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:07.722477 1324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:07.722477 1322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:07.722609 1316 server_base.cc:1034] running on GCE node
I20260924 00:14:07.722901 1316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:07.723158 1316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:07.724368 1316 hybrid_clock.cc:648] HybridClock initialized: now 1790208847724354 us; error 27 us; skew 500 ppm
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:07.727435 1316 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:07.727938 1316 webserver.cc:460] Webserver started at http://127.1.44.129:44673/ using document root <none> and password file <none>
I20260924 00:14:07.728183 1316 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:07.728248 1316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:07.728381 1316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:07.729422 1316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "336efb2eb57940538948f13b91f60173"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "4b31f927449153cf2e13b5563f1eef64"
server_key_iv: "29e4db9a61cd3edf2d53f6516a85117f"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.729789 1316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "336efb2eb57940538948f13b91f60173"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "4b31f927449153cf2e13b5563f1eef64"
server_key_iv: "29e4db9a61cd3edf2d53f6516a85117f"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.731068 1316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:14:07.731978 1331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.732192 1316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:07.732290 1316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "336efb2eb57940538948f13b91f60173"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "4b31f927449153cf2e13b5563f1eef64"
server_key_iv: "29e4db9a61cd3edf2d53f6516a85117f"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.732384 1316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:07.748396 1316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:07.750216 1316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:07.750380 1316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:07.750604 1316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:07.750900 1316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:07.750955 1316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.750996 1316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:07.751024 1316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.757352 1316 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:38143
I20260924 00:14:07.757395 1444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:38143 every 8 connection(s)
I20260924 00:14:07.757694 1316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:07.765369 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1316
I20260924 00:14:07.765447 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:07.765611 1202 external_mini_cluster.cc:1428] Setting key 611bd30d6ebb79e504399f7c1534c54e
I20260924 00:14:07.766922 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:07.758412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:55521 (local address 127.1.44.190:46749)
0924 00:14:07.758496 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:07.758498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:07.758670 (+ 172us) server_negotiation.cc:407] Connection header received
0924 00:14:07.759222 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:07.759223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:07.759240 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:07.759294 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:07.760222 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.760723 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.761753 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.761965 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.762050 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:07.764169 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:07.764170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:07.764170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:07.764176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:07.765500 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:07.765980 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:07.765982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:07.765982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:07.766044 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:07.766294 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:07.766295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:07.766295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:07.766415 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:07.766490 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:07.766751 (+ 261us) server_negotiation.cc:299] Negotiation successful
0924 00:14:07.766831 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:14:07.767141 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:07.758368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46749 (local address 127.1.44.129:55521)
0924 00:14:07.758541 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:07.758589 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:07.759086 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:07.759362 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:07.759366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:07.759933 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:07.760113 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:07.760117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.760822 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.760823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:07.761655 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:07.761657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.762059 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.762060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:07.762189 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:07.762690 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:07.762697 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:07.764087 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:07.765603 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:07.765610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:07.765612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:07.765888 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:07.766110 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:07.766111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:07.766111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:07.766226 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:07.766482 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:07.766488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:07.766623 (+ 135us) client_negotiation.cc:769] Sending connection context
0924 00:14:07.766731 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 00:14:07.766888 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260924 00:14:07.767668 1445 heartbeater.cc:347] Connected to a master server at 127.1.44.190:46749
I20260924 00:14:07.767781 1445 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:07.767970 1445 heartbeater.cc:511] Master 127.1.44.190:46749 requested a full tablet report, sending...
I20260924 00:14:07.768494 1254 ts_manager.cc:194] Registered new tserver with Master: 336efb2eb57940538948f13b91f60173 (127.1.44.129:38143)
I20260924 00:14:07.769109 1254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:55521
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:07.790221 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46749
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:07.872546 1450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:07.872742 1450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:07.872764 1450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:07.873816 1450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:07.873850 1450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:07.873876 1450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:07.875296 1450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46749
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:07.875537 1450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:07.875787 1450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:07.880759 1458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:07.880777 1456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:07.880951 1450 server_base.cc:1034] running on GCE node
W20260924 00:14:07.880779 1455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:07.881217 1450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:07.881466 1450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:07.882653 1450 hybrid_clock.cc:648] HybridClock initialized: now 1790208847882637 us; error 46 us; skew 500 ppm
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:07.885298 1450 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:07.885721 1450 webserver.cc:460] Webserver started at http://127.1.44.130:39229/ using document root <none> and password file <none>
I20260924 00:14:07.885943 1450 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:07.885987 1450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:07.886119 1450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:07.887143 1450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "de20017b936046cf8871fd85c237313a"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "e23a51d8e27906f1133deba58f6309be"
server_key_iv: "7035a92ad6cbf14a03fa98797c4da39f"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.887574 1450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "de20017b936046cf8871fd85c237313a"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "e23a51d8e27906f1133deba58f6309be"
server_key_iv: "7035a92ad6cbf14a03fa98797c4da39f"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.888854 1450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:07.889572 1465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.889757 1450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:07.889828 1450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "de20017b936046cf8871fd85c237313a"
format_stamp: "Formatted at 2026-09-24 00:14:07 on dist-test-slave-cm7b"
server_key: "e23a51d8e27906f1133deba58f6309be"
server_key_iv: "7035a92ad6cbf14a03fa98797c4da39f"
server_key_version: "encryptionkey@0"
I20260924 00:14:07.889914 1450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:07.901183 1450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:07.903484 1450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:07.903636 1450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:07.903853 1450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:07.904160 1450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:07.904218 1450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.904250 1450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:07.904271 1450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:07.910478 1450 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:37349
I20260924 00:14:07.910553 1578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:37349 every 8 connection(s)
I20260924 00:14:07.910857 1450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:07.914538 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1450
I20260924 00:14:07.914654 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:07.914816 1202 external_mini_cluster.cc:1428] Setting key c8107bf2c8532cdb3917c18fa5492394
Sep 24 00:14:07 dist-test-slave-cm7b krb5kdc[1209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208847, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:07.920504 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:07.911594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:53341 (local address 127.1.44.190:46749)
0924 00:14:07.911677 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:07.911679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:07.911863 (+ 184us) server_negotiation.cc:407] Connection header received
0924 00:14:07.912395 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:07.912396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:07.912408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:07.912459 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:07.913388 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.913894 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.914992 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.915300 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.915362 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:07.917629 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:07.917630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:07.917631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:07.917640 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:07.919131 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:07.919570 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:07.919571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:07.919572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:07.919627 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:07.919886 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:07.919887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:07.919887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:07.919996 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:07.920071 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:07.920344 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 00:14:07.920417 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:07.920684 1581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:07.911563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46749 (local address 127.1.44.130:53341)
0924 00:14:07.911739 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:07.911772 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:07.912262 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:07.912527 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:07.912536 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:07.913082 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:07.913263 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:07.913268 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.913987 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.913988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:07.914870 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:07.914872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:07.915393 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:07.915395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:07.915539 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:07.916124 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:07.916138 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:07.917515 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:07.919246 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:07.919252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:07.919254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:07.919478 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:07.919707 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:07.919708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:07.919709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:07.919816 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:07.920063 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:07.920068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:07.920209 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 00:14:07.920310 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:14:07.920459 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260924 00:14:07.921233 1579 heartbeater.cc:347] Connected to a master server at 127.1.44.190:46749
I20260924 00:14:07.921325 1579 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:07.921492 1579 heartbeater.cc:511] Master 127.1.44.190:46749 requested a full tablet report, sending...
I20260924 00:14:07.921901 1254 ts_manager.cc:194] Registered new tserver with Master: de20017b936046cf8871fd85c237313a (127.1.44.130:37349)
I20260924 00:14:07.922431 1254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:53341
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:07.939989 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46749
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:08.019764 1584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:08.019954 1584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:08.019977 1584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:08.021039 1584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:08.021073 1584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:08.021099 1584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:08.022416 1584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46749
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:08.022580 1584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:08.022739 1584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:08.027552 1590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:08.027563 1589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:08.027648 1584 server_base.cc:1034] running on GCE node
W20260924 00:14:08.027563 1592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:08.027915 1584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:08.028146 1584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:08.029277 1584 hybrid_clock.cc:648] HybridClock initialized: now 1790208848029257 us; error 38 us; skew 500 ppm
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208848, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:08.031699 1584 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:08.032153 1584 webserver.cc:460] Webserver started at http://127.1.44.131:39049/ using document root <none> and password file <none>
I20260924 00:14:08.032387 1584 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:08.032452 1584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:08.032585 1584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:08.033672 1584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "47afd01081ca43f2ac96c9d36c2dfeca"
format_stamp: "Formatted at 2026-09-24 00:14:08 on dist-test-slave-cm7b"
server_key: "ba5d8359810c30d3d8fccb9b8b8849dd"
server_key_iv: "1f6da0937f4e3a71b2328046361a2228"
server_key_version: "encryptionkey@0"
I20260924 00:14:08.034076 1584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "47afd01081ca43f2ac96c9d36c2dfeca"
format_stamp: "Formatted at 2026-09-24 00:14:08 on dist-test-slave-cm7b"
server_key: "ba5d8359810c30d3d8fccb9b8b8849dd"
server_key_iv: "1f6da0937f4e3a71b2328046361a2228"
server_key_version: "encryptionkey@0"
I20260924 00:14:08.035475 1584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:14:08.036141 1599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:08.036324 1584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:08.036418 1584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "47afd01081ca43f2ac96c9d36c2dfeca"
format_stamp: "Formatted at 2026-09-24 00:14:08 on dist-test-slave-cm7b"
server_key: "ba5d8359810c30d3d8fccb9b8b8849dd"
server_key_iv: "1f6da0937f4e3a71b2328046361a2228"
server_key_version: "encryptionkey@0"
I20260924 00:14:08.036505 1584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:08.051878 1584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:08.053771 1584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:08.053933 1584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:08.054160 1584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:08.054466 1584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:08.054519 1584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:08.054561 1584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:08.054589 1584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:08.060976 1584 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:40111
I20260924 00:14:08.061017 1712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:40111 every 8 connection(s)
I20260924 00:14:08.061378 1584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:08.064445 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1584
I20260924 00:14:08.064548 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:08.064750 1202 external_mini_cluster.cc:1428] Setting key 9077a973ab261af9f2d6e1b1a1a263f7
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208848, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:08.070974 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:08.062067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:50339 (local address 127.1.44.190:46749)
0924 00:14:08.062129 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:08.062131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:08.062356 (+ 225us) server_negotiation.cc:407] Connection header received
0924 00:14:08.062923 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:08.062943 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:08.062957 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:08.063010 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:08.064034 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.064594 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.065957 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.066140 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.066195 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:08.068263 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:08.068265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:08.068265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:08.068272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:08.069549 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:08.069987 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:08.069988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:08.069988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:08.070050 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:08.070314 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:08.070315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:08.070315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:08.070431 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:08.070504 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:08.070796 (+ 292us) server_negotiation.cc:299] Negotiation successful
0924 00:14:08.070881 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 00:14:08.071157 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:08.062056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46749 (local address 127.1.44.131:50339)
0924 00:14:08.062221 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:08.062267 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:08.062813 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:08.063080 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:08.063084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:08.063736 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:08.063932 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:08.063937 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.064676 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.064678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:08.065878 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:08.065881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.066209 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.066209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:08.066312 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:08.066752 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:08.066761 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:08.068143 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:08.069669 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:08.069675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:08.069677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:08.069907 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:08.070124 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:08.070125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:08.070125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:08.070235 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:08.070496 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:08.070502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:08.070650 (+ 148us) client_negotiation.cc:769] Sending connection context
0924 00:14:08.070763 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 00:14:08.070928 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260924 00:14:08.071606 1713 heartbeater.cc:347] Connected to a master server at 127.1.44.190:46749
I20260924 00:14:08.071713 1713 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:08.071890 1713 heartbeater.cc:511] Master 127.1.44.190:46749 requested a full tablet report, sending...
I20260924 00:14:08.072275 1254 ts_manager.cc:194] Registered new tserver with Master: 47afd01081ca43f2ac96c9d36c2dfeca (127.1.44.131:40111)
I20260924 00:14:08.072655 1254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:50339
I20260924 00:14:08.076877 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208848, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208848, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:08.101016 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:08.093116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42652 (local address 127.1.44.129:38143)
0924 00:14:08.093323 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:08.093327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:08.093338 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:14:08.093363 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:08.093365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:08.093378 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:08.093457 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:08.094048 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.094775 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.095585 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.095890 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.095952 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:08.098275 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:08.098282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:08.098282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:08.098288 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:08.099930 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:08.100280 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:08.100283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:08.100283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:08.100340 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:08.100559 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:08.100560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:08.100560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:08.100697 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:08.100808 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:08.100874 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 00:14:08.100934 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
W20260924 00:14:08.101449 1359 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42652
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:08.122157 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:08.115649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.1.44.129:38143)
0924 00:14:08.115693 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:08.115695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:08.115760 (+ 65us) server_negotiation.cc:407] Connection header received
0924 00:14:08.115801 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:08.115802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:08.115813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:08.115859 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:08.116398 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.116878 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.117591 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:08.117784 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:08.117847 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:08.120013 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:08.120014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:08.120014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:08.120022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:08.121176 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:08.121470 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:08.121471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:08.121471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:08.121516 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:08.121708 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:08.121709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:08.121710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:08.121811 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:08.121913 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:08.121977 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 00:14:08.122040 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:08.122706 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1316
I20260924 00:14:08.127449 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1450
I20260924 00:14:08.132361 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1584
I20260924 00:14:08.136502 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1222
2026-09-24T00:14:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1866 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1727](info): set up 2 sockets
Sep 24 00:14:08 dist-test-slave-cm7b krb5kdc[1727](info): commencing operation
krb5kdc: starting...
W20260924 00:14:09.166685 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:09.277931 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:41977
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:41977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:09.355923 1740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:09.356118 1740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:09.356139 1740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:09.357298 1740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:09.357339 1740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:09.357354 1740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:09.357366 1740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:09.357379 1740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:09.358757 1740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:41977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:41977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:09.358935 1740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:09.359091 1740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:09.363713 1745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:09.363722 1746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:09.363721 1748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:09.363919 1740 server_base.cc:1034] running on GCE node
I20260924 00:14:09.364148 1740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:09.364429 1740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:09.365579 1740 hybrid_clock.cc:648] HybridClock initialized: now 1790208849365571 us; error 32 us; skew 500 ppm
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:09.372334 1740 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:09.372788 1740 webserver.cc:460] Webserver started at http://127.1.44.190:37279/ using document root <none> and password file <none>
I20260924 00:14:09.373008 1740 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:09.373073 1740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:09.373214 1740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:09.374226 1740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "38c27732988b4d7db8beaafd3e62bfcc"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "5994326de2a1efc115790d364310cb74"
server_key_iv: "223fada3223287ae2e815a92a80b5c53"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.374565 1740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "38c27732988b4d7db8beaafd3e62bfcc"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "5994326de2a1efc115790d364310cb74"
server_key_iv: "223fada3223287ae2e815a92a80b5c53"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.375897 1740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:09.376569 1755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.376729 1740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.376830 1740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "38c27732988b4d7db8beaafd3e62bfcc"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "5994326de2a1efc115790d364310cb74"
server_key_iv: "223fada3223287ae2e815a92a80b5c53"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.376904 1740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:09.385913 1740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:09.387631 1740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:09.387782 1740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:09.391786 1740 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:41977
I20260924 00:14:09.391809 1807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:41977 every 8 connection(s)
I20260924 00:14:09.392132 1740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:09.392867 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:09.394982 1808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc: Bootstrap starting.
I20260924 00:14:09.395576 1808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:09.395859 1808 log.cc:824] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:09.396523 1808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc: No bootstrap required, opened a new log
I20260924 00:14:09.398029 1808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c27732988b4d7db8beaafd3e62bfcc" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41977 } }
I20260924 00:14:09.398190 1808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:09.398237 1808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c27732988b4d7db8beaafd3e62bfcc, State: Initialized, Role: FOLLOWER
I20260924 00:14:09.398388 1808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c27732988b4d7db8beaafd3e62bfcc" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41977 } }
I20260924 00:14:09.398469 1808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:09.398512 1808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:09.398559 1808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:09.399047 1808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c27732988b4d7db8beaafd3e62bfcc" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41977 } }
I20260924 00:14:09.399160 1808 leader_election.cc:302] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c27732988b4d7db8beaafd3e62bfcc; no voters:
I20260924 00:14:09.399353 1808 leader_election.cc:288] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:09.399402 1811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:09.399611 1808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:09.399616 1811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [term 1 LEADER]: Becoming Leader. State: Replica: 38c27732988b4d7db8beaafd3e62bfcc, State: Running, Role: LEADER
I20260924 00:14:09.399734 1811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c27732988b4d7db8beaafd3e62bfcc" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41977 } }
I20260924 00:14:09.400107 1813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c27732988b4d7db8beaafd3e62bfcc. Latest consensus state: current_term: 1 leader_uuid: "38c27732988b4d7db8beaafd3e62bfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c27732988b4d7db8beaafd3e62bfcc" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41977 } } }
I20260924 00:14:09.400355 1813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:09.400282 1812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c27732988b4d7db8beaafd3e62bfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c27732988b4d7db8beaafd3e62bfcc" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41977 } } }
I20260924 00:14:09.400537 1812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:09.400652 1818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:09.401140 1818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:09.401989 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1740
I20260924 00:14:09.402079 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:09.402240 1202 external_mini_cluster.cc:1428] Setting key 73be1847c88bc5eb3f53271c693ae15e
I20260924 00:14:09.402757 1818 catalog_manager.cc:1347] Generated new cluster ID: f13a359731a5427888e6a6dd25cb5cef
I20260924 00:14:09.402812 1818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:09.412199 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.402715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.1.44.190:41977)
0924 00:14:09.403557 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:09.403564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:09.403586 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:14:09.403678 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:09.403684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:09.403723 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:09.403852 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:09.404447 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.405371 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.406110 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.406506 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.406622 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.408900 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:09.408903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:09.408904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:09.408916 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:09.410566 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.410928 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.410932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.410932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.411029 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.411272 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.411274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.411274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.411523 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:09.411691 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:09.411795 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 00:14:09.412018 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":59,"threads_started":1}
I20260924 00:14:09.432937 1818 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:09.433485 1818 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:09.439309 1818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c27732988b4d7db8beaafd3e62bfcc: Generated new TSK 0
I20260924 00:14:09.439496 1818 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:09.472519 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41977
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:09.573436 1834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:09.573628 1834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:09.573652 1834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:09.574784 1834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:09.574826 1834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:09.574852 1834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:09.576339 1834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:09.576560 1834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:09.576789 1834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:09.582054 1842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:09.582069 1840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:09.582136 1834 server_base.cc:1034] running on GCE node
W20260924 00:14:09.582077 1839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:09.582552 1834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:09.582746 1834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:09.583900 1834 hybrid_clock.cc:648] HybridClock initialized: now 1790208849583889 us; error 29 us; skew 500 ppm
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:09.586313 1834 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:09.586751 1834 webserver.cc:460] Webserver started at http://127.1.44.129:45057/ using document root <none> and password file <none>
I20260924 00:14:09.586992 1834 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:09.587083 1834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:09.587293 1834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:09.588337 1834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "c208bedb143db4ac99430bee24c42171"
server_key_iv: "1173197c73d0c1b122eec3ef5bb3e1a0"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.588701 1834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "c208bedb143db4ac99430bee24c42171"
server_key_iv: "1173197c73d0c1b122eec3ef5bb3e1a0"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.590025 1834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:09.590768 1849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.590974 1834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:09.591071 1834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "c208bedb143db4ac99430bee24c42171"
server_key_iv: "1173197c73d0c1b122eec3ef5bb3e1a0"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.591197 1834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:09.598695 1834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:09.600798 1834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:09.600973 1834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:09.601200 1834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:09.601583 1834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:09.601637 1834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.601676 1834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:09.601706 1834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.607357 1834 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:42167
I20260924 00:14:09.607427 1962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:42167 every 8 connection(s)
I20260924 00:14:09.607750 1834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:09.617296 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.608522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:37345 (local address 127.1.44.190:41977)
0924 00:14:09.608613 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:09.608616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:09.608800 (+ 184us) server_negotiation.cc:407] Connection header received
0924 00:14:09.609790 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:09.609792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:09.609803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:09.609857 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:09.610505 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.611016 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.612061 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.612287 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.612375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.614490 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:09.614491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:09.614491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:09.614499 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:09.615932 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.616341 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.616342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.616343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.616406 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.616656 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.616657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.616657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.616775 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:09.616846 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:09.617117 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 00:14:09.617199 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 00:14:09.617491 1965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.608493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:41977 (local address 127.1.44.129:37345)
0924 00:14:09.608667 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:09.608712 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:09.609675 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:09.609941 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:09.609946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:09.610199 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:09.610394 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:09.610398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.611109 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.611110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:09.611966 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:09.611968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.612383 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.612384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:09.612511 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.613055 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:09.613063 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:09.614393 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:09.616032 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:09.616039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:09.616041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:09.616266 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:09.616473 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:09.616474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:09.616474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:09.616584 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:09.616849 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:09.616856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:09.616990 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 00:14:09.617095 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:14:09.617257 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260924 00:14:09.617656 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1834
I20260924 00:14:09.617750 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:09.617921 1202 external_mini_cluster.cc:1428] Setting key e82294f13e179e86b36921c40eee0b5b
I20260924 00:14:09.617961 1963 heartbeater.cc:347] Connected to a master server at 127.1.44.190:41977
I20260924 00:14:09.618075 1963 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:09.618268 1963 heartbeater.cc:511] Master 127.1.44.190:41977 requested a full tablet report, sending...
I20260924 00:14:09.618806 1772 ts_manager.cc:194] Registered new tserver with Master: 56ce7a07aaa44ec6b42b8dc0a533a52b (127.1.44.129:42167)
I20260924 00:14:09.619480 1772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:37345
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:09.642357 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41977
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:09.727324 1968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:09.727557 1968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:09.727594 1968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:09.728765 1968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:09.728834 1968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:09.728876 1968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:09.730363 1968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:09.730638 1968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:09.730875 1968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:09.735873 1973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:09.735890 1968 server_base.cc:1034] running on GCE node
W20260924 00:14:09.735854 1976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:09.735862 1974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:09.736261 1968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:09.736524 1968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:09.737663 1968 hybrid_clock.cc:648] HybridClock initialized: now 1790208849737655 us; error 37 us; skew 500 ppm
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:09.740175 1968 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:09.740584 1968 webserver.cc:460] Webserver started at http://127.1.44.130:44603/ using document root <none> and password file <none>
I20260924 00:14:09.740805 1968 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:09.740870 1968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:09.741015 1968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:09.742048 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "4a207869ee424982ae0fe8a361427fba"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "9bd63fb0a967d8c2778f89c1ca8e8816"
server_key_iv: "ee6eced13632f684ad4626c03576c107"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.742398 1968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "4a207869ee424982ae0fe8a361427fba"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "9bd63fb0a967d8c2778f89c1ca8e8816"
server_key_iv: "ee6eced13632f684ad4626c03576c107"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.743814 1968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:09.744565 1983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.744753 1968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:09.744853 1968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "4a207869ee424982ae0fe8a361427fba"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "9bd63fb0a967d8c2778f89c1ca8e8816"
server_key_iv: "ee6eced13632f684ad4626c03576c107"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.744923 1968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:09.781107 1968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:09.783563 1968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:09.783737 1968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:09.783986 1968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:09.784312 1968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:09.784366 1968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.784410 1968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:09.784435 1968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.789921 1968 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:34365
I20260924 00:14:09.790019 2096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:34365 every 8 connection(s)
I20260924 00:14:09.790342 1968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:09.797261 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 1968
I20260924 00:14:09.797340 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:09.797542 1202 external_mini_cluster.cc:1428] Setting key b1fc159a834df2e85da5a3ebe0a4a23c
I20260924 00:14:09.800302 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.791092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:51749 (local address 127.1.44.190:41977)
0924 00:14:09.791181 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:09.791184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:09.791424 (+ 240us) server_negotiation.cc:407] Connection header received
0924 00:14:09.792319 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:09.792320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:09.792329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:09.792409 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:09.792996 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.793482 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.794557 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.794737 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.794789 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.796956 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:09.796957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:09.796957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:09.796964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:09.798602 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.799093 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.799094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.799094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.799174 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.799504 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.799505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.799505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.799669 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:09.799755 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:09.800084 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 00:14:09.800176 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 00:14:09.800479 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.791054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:41977 (local address 127.1.44.130:51749)
0924 00:14:09.791304 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:09.791340 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:09.792231 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:09.792473 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:09.792476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:09.792730 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:09.792925 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:09.792929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.793559 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.793559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:09.794485 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:09.794487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.794805 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.794805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:09.794932 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.795534 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:09.795543 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:09.796871 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:09.798679 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:09.798686 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:09.798688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:09.799017 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:09.799281 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:09.799282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:09.799283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:09.799441 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:09.799756 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:09.799761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:09.799949 (+ 188us) client_negotiation.cc:769] Sending connection context
0924 00:14:09.800037 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 00:14:09.800239 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":105,"threads_started":1}
I20260924 00:14:09.801013 2097 heartbeater.cc:347] Connected to a master server at 127.1.44.190:41977
I20260924 00:14:09.801124 2097 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:09.801324 2097 heartbeater.cc:511] Master 127.1.44.190:41977 requested a full tablet report, sending...
I20260924 00:14:09.801723 1772 ts_manager.cc:194] Registered new tserver with Master: 4a207869ee424982ae0fe8a361427fba (127.1.44.130:34365)
I20260924 00:14:09.802103 1772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:51749
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:09.824249 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41977
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:09.905642 2102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:09.905834 2102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:09.905856 2102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:09.906929 2102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:09.906960 2102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:09.906985 2102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:09.908457 2102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41977
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:09.908689 2102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:09.908869 2102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:09.913609 2110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:09.913659 2107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:09.913615 2108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:09.913854 2102 server_base.cc:1034] running on GCE node
I20260924 00:14:09.914057 2102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:09.914305 2102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:09.915551 2102 hybrid_clock.cc:648] HybridClock initialized: now 1790208849915532 us; error 25 us; skew 500 ppm
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:09.917902 2102 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:09.918264 2102 webserver.cc:460] Webserver started at http://127.1.44.131:36475/ using document root <none> and password file <none>
I20260924 00:14:09.918491 2102 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:09.918535 2102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:09.918680 2102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:09.919730 2102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "c339793472684fb485ae5ef7143c4e7c"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "c5e434e02ee30e0bc94fdca83bcc256e"
server_key_iv: "44d2e9a4aec6e42409931115e3bc5f85"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.920075 2102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "c339793472684fb485ae5ef7143c4e7c"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "c5e434e02ee30e0bc94fdca83bcc256e"
server_key_iv: "44d2e9a4aec6e42409931115e3bc5f85"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.921329 2102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:09.922016 2117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.922184 2102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:09.922281 2102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "c339793472684fb485ae5ef7143c4e7c"
format_stamp: "Formatted at 2026-09-24 00:14:09 on dist-test-slave-cm7b"
server_key: "c5e434e02ee30e0bc94fdca83bcc256e"
server_key_iv: "44d2e9a4aec6e42409931115e3bc5f85"
server_key_version: "encryptionkey@0"
I20260924 00:14:09.922362 2102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:09.962149 2102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:09.964156 2102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:09.964322 2102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:09.964954 2102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:09.965281 2102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:09.965344 2102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.965380 2102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:09.965409 2102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:09.970927 2102 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:40477
I20260924 00:14:09.970999 2230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:40477 every 8 connection(s)
I20260924 00:14:09.971370 2102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
Sep 24 00:14:09 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:09.979583 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 2102
I20260924 00:14:09.979712 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:09.979894 1202 external_mini_cluster.cc:1428] Setting key efce1eca04c92421e365f68211e60f44
I20260924 00:14:09.980682 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.972121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:47383 (local address 127.1.44.190:41977)
0924 00:14:09.972166 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:09.972168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:09.972379 (+ 211us) server_negotiation.cc:407] Connection header received
0924 00:14:09.973337 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:09.973339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:09.973352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:09.973415 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:09.974021 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.974520 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.975633 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.975829 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.975914 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.977813 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:09.977814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:09.977814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:09.977820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:09.979119 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.979551 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.979552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.979553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.979657 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.979931 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.979932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.979932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.980059 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:09.980148 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:09.980519 (+ 371us) server_negotiation.cc:299] Negotiation successful
0924 00:14:09.980594 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:09.980867 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.972090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:41977 (local address 127.1.44.131:47383)
0924 00:14:09.972249 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:09.972293 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:09.973217 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:09.973485 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:09.973489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:09.973736 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:09.973922 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:09.973926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.974613 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.974614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:09.975545 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:09.975547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.975923 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.975924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:09.976048 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.976460 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:09.976467 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:09.977731 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:09.979191 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:09.979247 (+ 56us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:09.979248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:09.979465 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:09.979735 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:09.979736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:09.979736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:09.979859 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:09.980208 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:09.980213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:09.980381 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 00:14:09.980484 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 00:14:09.980629 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260924 00:14:09.981317 2231 heartbeater.cc:347] Connected to a master server at 127.1.44.190:41977
I20260924 00:14:09.981396 2231 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:09.981578 2231 heartbeater.cc:511] Master 127.1.44.190:41977 requested a full tablet report, sending...
I20260924 00:14:09.981917 1772 ts_manager.cc:194] Registered new tserver with Master: c339793472684fb485ae5ef7143c4e7c (127.1.44.131:40477)
I20260924 00:14:09.982275 1772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:47383
I20260924 00:14:09.991585 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:14:09.999095 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:09.993045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36762 (local address 127.1.44.190:41977)
0924 00:14:09.993117 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:09.993119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:09.993211 (+ 92us) server_negotiation.cc:407] Connection header received
0924 00:14:09.993261 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:09.993262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:09.993270 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:09.993358 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:09.993848 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.994574 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.995407 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:09.995605 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:09.995695 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:09.996761 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:09.996763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:09.996763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:09.996770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:09.997874 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.998255 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.998256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.998256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.998320 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:09.998528 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:09.998529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:09.998529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:09.998681 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:09.998797 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:09.998919 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 00:14:09.998993 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:14:10.001479 1772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:10.002938 1772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:10.009045 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.005289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46864 (local address 127.1.44.130:34365)
0924 00:14:10.005460 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.005465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.005477 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:14:10.005520 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.005523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.005537 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.005635 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:10.005799 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.006642 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.007852 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.008609 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.008685 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.008740 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.008921 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.008967 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260924 00:14:10.009065 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.005531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42462 (local address 127.1.44.129:42167)
0924 00:14:10.005673 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.005678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.005688 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:10.005730 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.005732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.005749 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.005995 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:10.006082 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.006832 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.008168 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.008749 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.008826 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.008884 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.008960 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.008996 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260924 00:14:10.009076 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.005199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:42167 (local address 127.0.0.1:42462)
0924 00:14:10.005452 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:10.005463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:10.005514 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:10.005863 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:10.005864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:10.005866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:10.005910 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.005911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.006935 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.006936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.008075 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.008078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.008828 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.008829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.008894 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.008897 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:10.008949 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 00:14:10.009005 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260924 00:14:10.009469 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.005189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:34365 (local address 127.0.0.1:46864)
0924 00:14:10.005337 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:10.005379 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:10.005448 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:10.005641 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:10.005644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:10.005647 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:10.005714 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.005716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.006754 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.006755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.007765 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.007768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.008700 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.008701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.008787 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.008806 (+ 19us) client_negotiation.cc:769] Sending connection context
0924 00:14:10.009353 (+ 547us) client_negotiation.cc:240] Negotiation successful
0924 00:14:10.009398 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260924 00:14:10.010011 1897 tablet_service.cc:1467] Processing CreateTablet for tablet d5aadbf1af444c2e83cff9267ecaace9 (DEFAULT_TABLE table=test-table [id=69af8a7b7e6e4d059df8566fbf39bcb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:10.010175 2031 tablet_service.cc:1467] Processing CreateTablet for tablet d5aadbf1af444c2e83cff9267ecaace9 (DEFAULT_TABLE table=test-table [id=69af8a7b7e6e4d059df8566fbf39bcb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:10.010314 1897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5aadbf1af444c2e83cff9267ecaace9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:10.010519 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.005386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:40477 (local address 127.0.0.1:33624)
0924 00:14:10.005779 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:10.005788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:10.005836 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:10.007017 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:10.007018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:10.007020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:10.007059 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.007060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.008287 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.008288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.009340 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.009343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.010285 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.010286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.010351 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.010356 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:14:10.010423 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 00:14:10.010459 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":62,"threads_started":1}
I20260924 00:14:10.010525 2031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5aadbf1af444c2e83cff9267ecaace9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:10.010567 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.005769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.1.44.131:40477)
0924 00:14:10.006806 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.006810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.006819 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:10.006853 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.006855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.006869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.006940 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:10.007139 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.008192 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.009433 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.010182 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.010271 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.010326 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.010438 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.010487 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":89,"threads_started":1}
I20260924 00:14:10.012912 2248 tablet_bootstrap.cc:492] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b: Bootstrap starting.
I20260924 00:14:10.013362 2249 tablet_bootstrap.cc:492] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: Bootstrap starting.
I20260924 00:14:10.013559 2165 tablet_service.cc:1467] Processing CreateTablet for tablet d5aadbf1af444c2e83cff9267ecaace9 (DEFAULT_TABLE table=test-table [id=69af8a7b7e6e4d059df8566fbf39bcb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:10.013671 2248 tablet_bootstrap.cc:654] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:10.013842 2165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5aadbf1af444c2e83cff9267ecaace9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:10.013986 2249 tablet_bootstrap.cc:654] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:10.014264 2248 log.cc:824] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:10.014565 2249 log.cc:824] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:10.015074 2248 tablet_bootstrap.cc:492] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b: No bootstrap required, opened a new log
I20260924 00:14:10.015127 2248 ts_tablet_manager.cc:1397] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:10.015424 2249 tablet_bootstrap.cc:492] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: No bootstrap required, opened a new log
I20260924 00:14:10.015549 2249 ts_tablet_manager.cc:1397] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:10.016408 2248 raft_consensus.cc:348] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.016543 2248 raft_consensus.cc:374] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:10.016566 2248 raft_consensus.cc:729] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56ce7a07aaa44ec6b42b8dc0a533a52b, State: Initialized, Role: FOLLOWER
I20260924 00:14:10.016685 2248 consensus_queue.cc:260] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.016963 2248 ts_tablet_manager.cc:1428] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:10.016974 1963 heartbeater.cc:503] Master 127.1.44.190:41977 was elected leader, sending a full tablet report...
I20260924 00:14:10.017315 2253 tablet_bootstrap.cc:492] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c: Bootstrap starting.
I20260924 00:14:10.017910 2253 tablet_bootstrap.cc:654] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:10.017925 2249 raft_consensus.cc:348] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.018085 2249 raft_consensus.cc:374] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:10.018131 2249 raft_consensus.cc:729] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a207869ee424982ae0fe8a361427fba, State: Initialized, Role: FOLLOWER
I20260924 00:14:10.018244 2249 consensus_queue.cc:260] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.018448 2253 log.cc:824] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:10.018522 2249 ts_tablet_manager.cc:1428] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:14:10.018536 2097 heartbeater.cc:503] Master 127.1.44.190:41977 was elected leader, sending a full tablet report...
I20260924 00:14:10.019074 2253 tablet_bootstrap.cc:492] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c: No bootstrap required, opened a new log
I20260924 00:14:10.019152 2253 ts_tablet_manager.cc:1397] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:10.020663 2253 raft_consensus.cc:348] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.020803 2253 raft_consensus.cc:374] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:10.020846 2253 raft_consensus.cc:729] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c339793472684fb485ae5ef7143c4e7c, State: Initialized, Role: FOLLOWER
I20260924 00:14:10.020951 2253 consensus_queue.cc:260] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.021221 2253 ts_tablet_manager.cc:1428] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:10.021229 2231 heartbeater.cc:503] Master 127.1.44.190:41977 was elected leader, sending a full tablet report...
I20260924 00:14:10.032421 2256 raft_consensus.cc:482] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:10.032492 2256 raft_consensus.cc:504] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.033206 2256 leader_election.cc:288] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a207869ee424982ae0fe8a361427fba (127.1.44.130:34365), 56ce7a07aaa44ec6b42b8dc0a533a52b (127.1.44.129:42167)
I20260924 00:14:10.036854 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.033433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:54165 (local address 127.1.44.129:42167)
0924 00:14:10.033489 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.033491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.033501 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:10.033533 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.033534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.033544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.033620 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:10.033791 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.034524 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.035949 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.036507 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.036582 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.036590 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.036759 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.036793 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:14:10.036852 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.033476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:34059 (local address 127.1.44.130:34365)
0924 00:14:10.033516 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.033518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.033609 (+ 91us) server_negotiation.cc:407] Connection header received
0924 00:14:10.033690 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.033691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.033699 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.033741 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:10.033879 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.034547 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.035988 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.036506 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.036582 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.036590 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.036756 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.036791 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:10.036863 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.033382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:42167 (local address 127.1.44.131:54165)
0924 00:14:10.033437 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:10.033449 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:10.033488 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:10.033635 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:10.033635 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:10.033637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:10.033709 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.033711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.034626 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.034627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.035857 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.035859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.036587 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.036588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.036670 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.036674 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:10.036734 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 00:14:10.036779 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":11}
I20260924 00:14:10.036875 2257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.033382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:34365 (local address 127.1.44.131:34059)
0924 00:14:10.033536 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:10.033544 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:10.033619 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:10.033761 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:10.033762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:10.033763 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:10.033821 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.033823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.034626 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.034627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.035857 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:10.035859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.036587 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.036588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:10.036675 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.036678 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:10.036734 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 00:14:10.036779 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":60,"thread_start_us":88,"threads_started":1}
I20260924 00:14:10.037205 1917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5aadbf1af444c2e83cff9267ecaace9" candidate_uuid: "c339793472684fb485ae5ef7143c4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" is_pre_election: true
I20260924 00:14:10.037346 1917 raft_consensus.cc:2454] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c339793472684fb485ae5ef7143c4e7c in term 0.
I20260924 00:14:10.037369 2051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5aadbf1af444c2e83cff9267ecaace9" candidate_uuid: "c339793472684fb485ae5ef7143c4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a207869ee424982ae0fe8a361427fba" is_pre_election: true
I20260924 00:14:10.037513 2051 raft_consensus.cc:2454] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c339793472684fb485ae5ef7143c4e7c in term 0.
I20260924 00:14:10.037734 2121 leader_election.cc:302] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a207869ee424982ae0fe8a361427fba, c339793472684fb485ae5ef7143c4e7c; no voters:
I20260924 00:14:10.037847 2256 raft_consensus.cc:2781] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:10.037896 2256 raft_consensus.cc:482] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:10.037917 2256 raft_consensus.cc:3037] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:10.038487 2256 raft_consensus.cc:504] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.038612 2256 leader_election.cc:288] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [CANDIDATE]: Term 1 election: Requested vote from peers 4a207869ee424982ae0fe8a361427fba (127.1.44.130:34365), 56ce7a07aaa44ec6b42b8dc0a533a52b (127.1.44.129:42167)
I20260924 00:14:10.038730 2051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5aadbf1af444c2e83cff9267ecaace9" candidate_uuid: "c339793472684fb485ae5ef7143c4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a207869ee424982ae0fe8a361427fba"
I20260924 00:14:10.038749 1917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5aadbf1af444c2e83cff9267ecaace9" candidate_uuid: "c339793472684fb485ae5ef7143c4e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b"
I20260924 00:14:10.038820 2051 raft_consensus.cc:3037] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:10.038817 1917 raft_consensus.cc:3037] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:10.039516 2051 raft_consensus.cc:2454] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c339793472684fb485ae5ef7143c4e7c in term 1.
I20260924 00:14:10.039547 1917 raft_consensus.cc:2454] T d5aadbf1af444c2e83cff9267ecaace9 P 56ce7a07aaa44ec6b42b8dc0a533a52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c339793472684fb485ae5ef7143c4e7c in term 1.
I20260924 00:14:10.039731 2121 leader_election.cc:302] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a207869ee424982ae0fe8a361427fba, c339793472684fb485ae5ef7143c4e7c; no voters:
I20260924 00:14:10.039836 2256 raft_consensus.cc:2781] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:10.039947 2256 raft_consensus.cc:686] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [term 1 LEADER]: Becoming Leader. State: Replica: c339793472684fb485ae5ef7143c4e7c, State: Running, Role: LEADER
I20260924 00:14:10.040042 2256 consensus_queue.cc:237] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } }
I20260924 00:14:10.040705 1772 catalog_manager.cc:5495] T d5aadbf1af444c2e83cff9267ecaace9 P c339793472684fb485ae5ef7143c4e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to c339793472684fb485ae5ef7143c4e7c (127.1.44.131). New cstate: current_term: 1 leader_uuid: "c339793472684fb485ae5ef7143c4e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a207869ee424982ae0fe8a361427fba" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56ce7a07aaa44ec6b42b8dc0a533a52b" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 42167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c339793472684fb485ae5ef7143c4e7c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40477 } health_report { overall_health: HEALTHY } } }
W20260924 00:14:10.040832 2098 tablet.cc:2369] T d5aadbf1af444c2e83cff9267ecaace9 P 4a207869ee424982ae0fe8a361427fba: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:10.051466 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.044659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.1.44.129:42167)
0924 00:14:10.044699 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.044701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.044799 (+ 98us) server_negotiation.cc:407] Connection header received
0924 00:14:10.044834 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.044835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.044842 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.044888 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:10.045418 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.045843 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.046524 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.046692 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.046742 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.049064 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:10.049071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:10.049072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:10.049077 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:10.050477 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:10.050774 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:10.050776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:10.050776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:10.050833 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:10.050988 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:10.050989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:10.050989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:10.051098 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:10.051252 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.051316 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.051373 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208850, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208850, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:10.072912 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.066355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.1.44.129:42167)
0924 00:14:10.066429 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.066431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.066540 (+ 109us) server_negotiation.cc:407] Connection header received
0924 00:14:10.066619 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.066620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.066629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.066702 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:10.067306 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.067747 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.068423 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.068593 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.068640 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.070658 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:10.070660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:10.070660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:10.070665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:10.072039 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:10.072317 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:10.072317 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:10.072318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:10.072362 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:10.072500 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:10.072501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:10.072501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:10.072572 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:10.072635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.072766 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.072817 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[1727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[1727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:10.094120 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:10.087113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.1.44.129:42167)
0924 00:14:10.087251 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:10.087254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:10.087263 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:10.087346 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:10.087346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:10.087354 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:10.087400 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:10.087878 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.088360 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.089114 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:10.089307 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:10.089366 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:10.091646 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:10.091648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:10.091648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:10.091656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:10.092960 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:10.093323 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:10.093324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:10.093324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:10.093380 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:10.093575 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:10.093575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:10.093576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:10.093696 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:10.093795 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:10.093945 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 00:14:10.094000 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
W20260924 00:14:10.094614 1877 tablet_service.cc:2908] Rejecting scan request for tablet d5aadbf1af444c2e83cff9267ecaace9: Uninitialized: clean time has not yet been initialized
I20260924 00:14:10.095423 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1834
I20260924 00:14:10.101114 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1968
I20260924 00:14:10.105655 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 2102
I20260924 00:14:10.109930 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 1740
2026-09-24T00:14:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1973 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[2272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[2272](info): set up 2 sockets
Sep 24 00:14:10 dist-test-slave-cm7b krb5kdc[2272](info): commencing operation
krb5kdc: starting...
W20260924 00:14:11.146694 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:11.258036 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34137
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:11.336403 2285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:11.336630 2285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:11.336665 2285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:11.337769 2285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:11.337831 2285 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 00:14:11.337853 2285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:11.337875 2285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:11.337903 2285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:11.337929 2285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:11.339391 2285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34137
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:11.339663 2285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:11.339882 2285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:11.344717 2291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:11.344710 2293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:11.344748 2285 server_base.cc:1034] running on GCE node
W20260924 00:14:11.344710 2290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:11.345237 2285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:11.345515 2285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:11.346686 2285 hybrid_clock.cc:648] HybridClock initialized: now 1790208851346666 us; error 32 us; skew 500 ppm
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:11.349325 2285 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:11.349684 2285 webserver.cc:460] Webserver started at http://127.1.44.190:45585/ using document root <none> and password file <none>
I20260924 00:14:11.349892 2285 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:11.349934 2285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:11.350019 2285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:11.350987 2285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "549c37ee166418074d13323387ab074e"
server_key_iv: "750bece35349e335cf36a70e67dc9d8e"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.351356 2285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "549c37ee166418074d13323387ab074e"
server_key_iv: "750bece35349e335cf36a70e67dc9d8e"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.352614 2285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:11.353235 2300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.353423 2285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:11.353499 2285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "549c37ee166418074d13323387ab074e"
server_key_iv: "750bece35349e335cf36a70e67dc9d8e"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.353550 2285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:11.372255 2285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:11.374073 2285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:11.374176 2285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:11.378266 2285 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34137
I20260924 00:14:11.378382 2362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34137 every 8 connection(s)
I20260924 00:14:11.378654 2285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:11.379333 2363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:11.381400 2363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: Bootstrap starting.
I20260924 00:14:11.381953 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 2285
I20260924 00:14:11.382037 2363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:11.382050 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:11.382241 1202 external_mini_cluster.cc:1428] Setting key 7eb61dc43c4e322d67391819ad812d64
I20260924 00:14:11.382357 2363 log.cc:824] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:11.383095 2363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: No bootstrap required, opened a new log
I20260924 00:14:11.384860 2363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34137 } }
I20260924 00:14:11.384986 2363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:11.385057 2363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b1c66a4aad4b8cbfd40d3db8b3f37f, State: Initialized, Role: FOLLOWER
I20260924 00:14:11.385188 2363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34137 } }
I20260924 00:14:11.385286 2363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:11.385329 2363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:11.385376 2363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:11.385993 2363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34137 } }
I20260924 00:14:11.386096 2363 leader_election.cc:302] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80b1c66a4aad4b8cbfd40d3db8b3f37f; no voters:
I20260924 00:14:11.386240 2363 leader_election.cc:288] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:11.386301 2368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:11.386461 2363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:11.386468 2368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [term 1 LEADER]: Becoming Leader. State: Replica: 80b1c66a4aad4b8cbfd40d3db8b3f37f, State: Running, Role: LEADER
I20260924 00:14:11.386648 2368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34137 } }
I20260924 00:14:11.387055 2370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80b1c66a4aad4b8cbfd40d3db8b3f37f. Latest consensus state: current_term: 1 leader_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34137 } } }
I20260924 00:14:11.387138 2370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:11.387382 2369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80b1c66a4aad4b8cbfd40d3db8b3f37f" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34137 } } }
I20260924 00:14:11.387498 2369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:11.387650 2377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:11.388087 2377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:11.389539 2377 catalog_manager.cc:1347] Generated new cluster ID: e4fa3ef41a054140a4c37544e1742612
I20260924 00:14:11.389943 2377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:11.393627 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.382745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35426 (local address 127.1.44.190:34137)
0924 00:14:11.382881 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.382895 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.382919 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:14:11.383378 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.383385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.383430 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.383541 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.384265 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.385587 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.386629 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.387319 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.387398 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.390167 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.390171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.390172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.390182 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.392016 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.392429 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.392432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.392432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.392522 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.392731 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.392732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.392732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.392986 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.393111 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.393314 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.393433 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
I20260924 00:14:11.400388 2377 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:11.401044 2377 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:11.403558 2377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: Generated new TSK 0
I20260924 00:14:11.403707 2377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:11.433661 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34137
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:11.513160 2389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:11.513386 2389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:11.513425 2389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:11.514583 2389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:11.514647 2389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:11.514684 2389 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:11.514740 2389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:11.516227 2389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:34137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:11.516500 2389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:11.516716 2389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:11.521502 2397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:11.521452 2394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:11.521452 2395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:11.521526 2389 server_base.cc:1034] running on GCE node
I20260924 00:14:11.521903 2389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:11.522174 2389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:11.523341 2389 hybrid_clock.cc:648] HybridClock initialized: now 1790208851523324 us; error 29 us; skew 500 ppm
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:11.525694 2389 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:11.526141 2389 webserver.cc:460] Webserver started at http://127.1.44.129:42667/ using document root <none> and password file <none>
I20260924 00:14:11.526379 2389 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:11.526446 2389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:11.526577 2389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:11.527638 2389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "e1fbd2ee570b460aa078a66344293252"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "cdcc3265691e67e22c1fd09db2c2d636"
server_key_iv: "b2022439079fb31c4c902b13aade02c6"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.527985 2389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "e1fbd2ee570b460aa078a66344293252"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "cdcc3265691e67e22c1fd09db2c2d636"
server_key_iv: "b2022439079fb31c4c902b13aade02c6"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.529268 2389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:11.529968 2404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.530143 2389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:11.530236 2389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "e1fbd2ee570b460aa078a66344293252"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "cdcc3265691e67e22c1fd09db2c2d636"
server_key_iv: "b2022439079fb31c4c902b13aade02c6"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.530328 2389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:11.547796 2389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:11.549553 2389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:11.549705 2389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:11.550209 2389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:11.550276 2389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.550318 2389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:11.550346 2389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:11.557303 2389 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:36825
I20260924 00:14:11.557657 2389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:11.557636 2519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:36825 every 8 connection(s)
I20260924 00:14:11.558126 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 2389
I20260924 00:14:11.558223 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:11.558357 1202 external_mini_cluster.cc:1428] Setting key e7e6184f43344dc80635fab798e8fc1c
I20260924 00:14:11.560624 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.550675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:52441 (local address 127.1.44.190:34137)
0924 00:14:11.550749 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.550752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.550948 (+ 196us) server_negotiation.cc:407] Connection header received
0924 00:14:11.551431 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.551433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.551446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.551532 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.552119 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.552619 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.554180 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.554548 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.554636 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.557081 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.557082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.557082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.557089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.559268 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.559649 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.559650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.559651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.559717 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.559942 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.559943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.559944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.560085 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.560153 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.560496 (+ 343us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.560561 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 00:14:11.560875 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.550610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.1.44.129:52441)
0924 00:14:11.550840 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.550866 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.551335 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.551579 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.551582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.551801 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:11.552026 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.552031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.552713 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.552714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.554087 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.554089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.554658 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.554659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.554788 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.555314 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:11.555329 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:11.556951 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:11.559363 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.559369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.559376 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.559575 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.559791 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.559792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.559792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.559881 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.560148 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:11.560152 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:11.560349 (+ 197us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.560467 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.560630 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
I20260924 00:14:11.561404 2520 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34137
I20260924 00:14:11.561532 2520 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:11.561745 2520 heartbeater.cc:511] Master 127.1.44.190:34137 requested a full tablet report, sending...
I20260924 00:14:11.562573 2316 ts_manager.cc:194] Registered new tserver with Master: e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825)
I20260924 00:14:11.563565 2316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:52441
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:14:11.568917 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.562507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:37247 (local address 127.1.44.190:34137)
0924 00:14:11.562541 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.562543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.563400 (+ 857us) server_negotiation.cc:407] Connection header received
0924 00:14:11.563466 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.563467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.563477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.563531 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.563957 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.564430 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.565147 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.565347 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.565431 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.566355 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.566356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.566357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.566363 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.567779 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.568139 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.568140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.568141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.568196 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.568398 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.568399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.568400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.568507 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.568581 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.568748 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.568822 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:11.568931 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.562349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.1.44.129:37247)
0924 00:14:11.563321 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.563332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.563413 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.563537 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.563538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.563789 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:11.563865 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.563867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.564530 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.564531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.565050 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.565051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.565431 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.565432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.565507 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.565959 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:11.565964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:11.566248 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:11.567879 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.567879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.567880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.568053 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.568266 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.568267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.568268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.568324 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.568580 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:11.568580 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:11.568630 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.568718 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.568822 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":58,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:11.583804 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34137
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:11.664994 2529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:11.665218 2529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:11.665242 2529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:11.666373 2529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:11.666409 2529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:11.666429 2529 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:11.666446 2529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:11.667883 2529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:34137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:11.668062 2529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:11.668231 2529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:11.673210 2537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:11.673238 2535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:11.673303 2529 server_base.cc:1034] running on GCE node
W20260924 00:14:11.673446 2534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:11.673662 2529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:11.673938 2529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:11.675115 2529 hybrid_clock.cc:648] HybridClock initialized: now 1790208851675091 us; error 35 us; skew 500 ppm
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:11.677644 2529 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:11.678061 2529 webserver.cc:460] Webserver started at http://127.1.44.130:38497/ using document root <none> and password file <none>
I20260924 00:14:11.678287 2529 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:11.678357 2529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:11.678509 2529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:11.679620 2529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "5b7978055b1f1f6198cd470447d90ebf"
server_key_iv: "69ff0d8ecd92c0343a04789db7456ea4"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.679992 2529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "5b7978055b1f1f6198cd470447d90ebf"
server_key_iv: "69ff0d8ecd92c0343a04789db7456ea4"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.681299 2529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:11.681998 2544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.682178 2529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:11.682283 2529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "5b7978055b1f1f6198cd470447d90ebf"
server_key_iv: "69ff0d8ecd92c0343a04789db7456ea4"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.682358 2529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:11.708724 2529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:11.711306 2529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:11.711468 2529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:11.712020 2529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:11.712083 2529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.712117 2529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:11.712146 2529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.719326 2529 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:37151
I20260924 00:14:11.719398 2659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:37151 every 8 connection(s)
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:11.719703 2529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:11.722740 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.712475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48239 (local address 127.1.44.190:34137)
0924 00:14:11.712547 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.712550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.712825 (+ 275us) server_negotiation.cc:407] Connection header received
0924 00:14:11.713576 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.713578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.713591 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.713668 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.714627 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.715467 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.717103 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.717317 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.717409 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.719930 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.719932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.719932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.719940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.721390 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.721800 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.721801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.721802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.721862 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.722112 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.722113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.722113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.722226 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.722301 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.722575 (+ 274us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.722650 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 00:14:11.722944 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.712451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.1.44.130:48239)
0924 00:14:11.712663 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.712722 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.713443 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.713773 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.713777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.714194 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:11.714521 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.714529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.715542 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.715543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.717008 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.717011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.717418 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.717420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.717591 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.718167 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:11.718175 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:11.719839 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:11.721492 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.721498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.721500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.721723 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.721930 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.721931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.721931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.722043 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.722292 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:11.722298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:11.722437 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.722548 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.722704 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260924 00:14:11.723435 2660 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34137
I20260924 00:14:11.723541 2660 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:11.723712 2660 heartbeater.cc:511] Master 127.1.44.190:34137 requested a full tablet report, sending...
I20260924 00:14:11.724117 2317 ts_manager.cc:194] Registered new tserver with Master: 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151)
I20260924 00:14:11.724609 2317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:48239
I20260924 00:14:11.728741 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 2529
I20260924 00:14:11.728813 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:11.729025 1202 external_mini_cluster.cc:1428] Setting key 7153522f7135354bb2e76d2e6df32495
I20260924 00:14:11.729883 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.724469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:49229 (local address 127.1.44.190:34137)
0924 00:14:11.724512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.724514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.724647 (+ 133us) server_negotiation.cc:407] Connection header received
0924 00:14:11.724672 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.724673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.724683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.724750 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.725083 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.725546 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.726242 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.726448 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.726533 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.727376 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.727377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.727378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.727384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.728642 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.729025 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.729026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.729027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.729098 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.729340 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.729340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.729340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.729478 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.729560 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.729714 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.729785 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:11.729892 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.724409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.1.44.130:49229)
0924 00:14:11.724564 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.724587 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.724635 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.724757 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.724757 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.724928 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:11.724999 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.725001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.725643 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.725644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.726157 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.726158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.726535 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.726536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.726608 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.726953 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:11.726957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:11.727266 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:11.728727 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.728728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.728728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.728931 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.729174 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.729175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.729176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.729252 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.729551 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:11.729552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:11.729597 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.729688 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.729787 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":6,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:11.753790 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34137
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:11.831840 2669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:11.832026 2669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:11.832050 2669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:11.833294 2669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:11.833334 2669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:11.833355 2669 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:11.833410 2669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:11.834833 2669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:34137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:11.835079 2669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:11.835319 2669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:11.840006 2674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:11.840016 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:11.840054 2677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:11.840224 2669 server_base.cc:1034] running on GCE node
I20260924 00:14:11.840415 2669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:11.840595 2669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:11.841760 2669 hybrid_clock.cc:648] HybridClock initialized: now 1790208851841740 us; error 32 us; skew 500 ppm
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:11.844118 2669 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:11.844550 2669 webserver.cc:460] Webserver started at http://127.1.44.131:33601/ using document root <none> and password file <none>
I20260924 00:14:11.844735 2669 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:11.844779 2669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:11.844862 2669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:11.845801 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "de6f964b971543f49ced0002b3b44660"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "e1d804ad7b0a5168eaccf7ad39004e96"
server_key_iv: "f59d69009a23db6ad2fa92d464b5f5d3"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.846082 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "de6f964b971543f49ced0002b3b44660"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "e1d804ad7b0a5168eaccf7ad39004e96"
server_key_iv: "f59d69009a23db6ad2fa92d464b5f5d3"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.847316 2669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:11.847970 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.848131 2669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:11.848213 2669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "de6f964b971543f49ced0002b3b44660"
format_stamp: "Formatted at 2026-09-24 00:14:11 on dist-test-slave-cm7b"
server_key: "e1d804ad7b0a5168eaccf7ad39004e96"
server_key_iv: "f59d69009a23db6ad2fa92d464b5f5d3"
server_key_version: "encryptionkey@0"
I20260924 00:14:11.848273 2669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:11.870801 2669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:11.872692 2669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:11.872814 2669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:11.873237 2669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:11.873304 2669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:11.873351 2669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:11.873376 2669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:11.880324 2669 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:36557
I20260924 00:14:11.880705 2799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:36557 every 8 connection(s)
I20260924 00:14:11.880726 2669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:11.882784 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.873751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:32789 (local address 127.1.44.190:34137)
0924 00:14:11.873791 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.873793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.874058 (+ 265us) server_negotiation.cc:407] Connection header received
0924 00:14:11.874581 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.874582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.874594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.874661 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.875504 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.876291 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.877367 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.877594 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.877672 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.879879 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.879880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.879880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.879888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.881440 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.881835 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.881836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.881836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.881892 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.882162 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.882163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.882163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.882282 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.882357 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.882614 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.882697 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:11.882959 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.873763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.1.44.131:32789)
0924 00:14:11.873943 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.873976 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.874475 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.874739 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.874744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.875088 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:11.875416 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.875424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.876427 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.876428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.877257 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.877259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.877705 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.877706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.877826 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.878365 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:11.878374 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:11.879784 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:11.881532 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.881538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.881539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.881753 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.881975 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.881976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.881976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.882077 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.882349 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:11.882355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:11.882488 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.882594 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.882739 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260924 00:14:11.883443 2800 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34137
I20260924 00:14:11.883523 2800 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:11.883704 2800 heartbeater.cc:511] Master 127.1.44.190:34137 requested a full tablet report, sending...
I20260924 00:14:11.884059 2316 ts_manager.cc:194] Registered new tserver with Master: de6f964b971543f49ced0002b3b44660 (127.1.44.131:36557)
I20260924 00:14:11.884487 2316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:32789
I20260924 00:14:11.888161 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 2669
I20260924 00:14:11.888264 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:11.888456 1202 external_mini_cluster.cc:1428] Setting key cbf22e8751207b42c0e6dd87132a64bc
I20260924 00:14:11.889523 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:14:11.889973 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.884508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:49799 (local address 127.1.44.190:34137)
0924 00:14:11.884545 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.884547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.884640 (+ 93us) server_negotiation.cc:407] Connection header received
0924 00:14:11.884698 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.884698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.884707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.884776 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.885096 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.885539 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.886319 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.886491 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.886549 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.887380 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.887381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.887382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.887387 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.888860 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.889204 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.889205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.889205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.889273 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.889471 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.889473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.889473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.889597 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.889659 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.889814 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.889884 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:11.889986 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.884420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.1.44.131:49799)
0924 00:14:11.884578 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.884590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.884651 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.884767 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.884768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.884934 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:11.884992 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.884993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.885625 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.885626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.886250 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.886252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.886557 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.886557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.886610 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.886969 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:11.886972 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:11.887323 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:11.888945 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.888946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.888946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.889140 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.889354 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:11.889356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:11.889356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:11.889423 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:11.889652 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:11.889653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:11.889696 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.889786 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.889887 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:14:11 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:11.920745 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.913191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.1.44.190:34137)
0924 00:14:11.913267 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.913269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.913284 (+ 15us) server_negotiation.cc:407] Connection header received
0924 00:14:11.913306 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.913307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.913314 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.913392 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:11.913993 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.914596 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.915813 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.915996 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.916058 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.918460 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:11.918461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:11.918462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:11.918468 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:11.919869 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.920156 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.920156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.920157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.920202 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:11.920366 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:11.920366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:11.920366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:11.920455 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:11.920519 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.920600 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.920652 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 00:14:11.922220 2316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:11.923887 2316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:11.930389 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.926000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37151 (local address 127.0.0.1:33726)
0924 00:14:11.926141 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.926180 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.926255 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.926726 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.926729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.926731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:11.926796 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.926798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.928056 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.928057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.929369 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.929371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.930048 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.930049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.930142 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.930155 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.930228 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.930281 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":1,"thread_start_us":59,"threads_started":1}
I20260924 00:14:11.930397 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.926355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.1.44.130:37151)
0924 00:14:11.926512 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.926517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.926528 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:14:11.926567 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.926570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.926587 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.926874 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:11.926929 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.927978 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.929473 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.929959 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.930050 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.930113 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.930263 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.930300 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260924 00:14:11.930395 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.926004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36825 (local address 127.0.0.1:41404)
0924 00:14:11.926616 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.926625 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.926674 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.926928 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.926929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.926930 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:11.926973 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.926974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.928277 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.928277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.929369 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.929371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.930064 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.930065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.930145 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.930157 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.930258 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.930300 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":7,"thread_start_us":78,"threads_started":1}
I20260924 00:14:11.930864 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.926177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41404 (local address 127.1.44.129:36825)
0924 00:14:11.926572 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.926576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.926741 (+ 165us) server_negotiation.cc:407] Connection header received
0924 00:14:11.926781 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.926784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.926797 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.926866 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:11.927037 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.928182 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.929466 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.929959 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.930045 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.930109 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.930726 (+ 617us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.930770 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":77,"threads_started":1}
I20260924 00:14:11.931141 2594 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0c5492f5354f9f885c3df168cc00af (DEFAULT_TABLE table=test-table [id=e28aec49ceea4cfe9b8d006177d1c80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:11.931547 2594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0c5492f5354f9f885c3df168cc00af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:11.931450 2454 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0c5492f5354f9f885c3df168cc00af (DEFAULT_TABLE table=test-table [id=e28aec49ceea4cfe9b8d006177d1c80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:11.931740 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.926303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.1.44.131:36557)
0924 00:14:11.926466 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.926471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.927163 (+ 692us) server_negotiation.cc:407] Connection header received
0924 00:14:11.927275 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.927278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.927294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.927377 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:11.927486 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.928458 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.930145 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.931393 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.931470 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.931520 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.931608 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.931647 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260924 00:14:11.931758 2454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0c5492f5354f9f885c3df168cc00af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:11.931948 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.926212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:36557 (local address 127.0.0.1:36850)
0924 00:14:11.927066 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.927075 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.927120 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.927387 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.927387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.927389 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:11.927425 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.927427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.928543 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.928544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.929376 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.929379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.931473 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.931474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.931535 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.931539 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.931687 (+ 148us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.931730 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":76,"threads_started":1}
I20260924 00:14:11.935348 2823 tablet_bootstrap.cc:492] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Bootstrap starting.
I20260924 00:14:11.935957 2823 tablet_bootstrap.cc:654] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:11.935882 2734 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0c5492f5354f9f885c3df168cc00af (DEFAULT_TABLE table=test-table [id=e28aec49ceea4cfe9b8d006177d1c80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:11.936175 2734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0c5492f5354f9f885c3df168cc00af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:11.936292 2823 log.cc:824] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:11.936602 2824 tablet_bootstrap.cc:492] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Bootstrap starting.
I20260924 00:14:11.937099 2824 tablet_bootstrap.cc:654] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:11.937498 2824 log.cc:824] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:11.937994 2823 tablet_bootstrap.cc:492] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: No bootstrap required, opened a new log
I20260924 00:14:11.938068 2823 ts_tablet_manager.cc:1397] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 00:14:11.938203 2824 tablet_bootstrap.cc:492] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: No bootstrap required, opened a new log
I20260924 00:14:11.938264 2824 ts_tablet_manager.cc:1397] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:11.939079 2827 tablet_bootstrap.cc:492] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Bootstrap starting.
I20260924 00:14:11.939849 2827 tablet_bootstrap.cc:654] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:11.940145 2827 log.cc:824] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:11.940244 2823 raft_consensus.cc:348] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.940418 2823 raft_consensus.cc:374] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:11.940471 2823 raft_consensus.cc:729] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3c74afaa7047dd9ee98e2a2d846cc5, State: Initialized, Role: FOLLOWER
I20260924 00:14:11.940454 2824 raft_consensus.cc:348] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.940598 2824 raft_consensus.cc:374] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:11.940641 2824 raft_consensus.cc:729] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fbd2ee570b460aa078a66344293252, State: Initialized, Role: FOLLOWER
I20260924 00:14:11.940608 2823 consensus_queue.cc:260] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.940784 2824 consensus_queue.cc:260] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.940832 2827 tablet_bootstrap.cc:492] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: No bootstrap required, opened a new log
I20260924 00:14:11.940876 2823 ts_tablet_manager.cc:1428] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:11.940892 2827 ts_tablet_manager.cc:1397] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:11.941004 2660 heartbeater.cc:503] Master 127.1.44.190:34137 was elected leader, sending a full tablet report...
I20260924 00:14:11.941030 2824 ts_tablet_manager.cc:1428] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:11.941143 2520 heartbeater.cc:503] Master 127.1.44.190:34137 was elected leader, sending a full tablet report...
I20260924 00:14:11.942605 2827 raft_consensus.cc:348] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.942744 2827 raft_consensus.cc:374] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:11.942790 2827 raft_consensus.cc:729] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6f964b971543f49ced0002b3b44660, State: Initialized, Role: FOLLOWER
I20260924 00:14:11.942898 2827 consensus_queue.cc:260] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.943145 2827 ts_tablet_manager.cc:1428] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:11.943245 2800 heartbeater.cc:503] Master 127.1.44.190:34137 was elected leader, sending a full tablet report...
I20260924 00:14:11.969298 2831 raft_consensus.cc:482] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:11.969404 2831 raft_consensus.cc:504] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.969707 2831 leader_election.cc:288] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825), 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151)
W20260924 00:14:11.970350 2661 tablet.cc:2369] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:11.972690 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.969913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:38507 (local address 127.1.44.129:36825)
0924 00:14:11.969953 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.969956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.970034 (+ 78us) server_negotiation.cc:407] Connection header received
0924 00:14:11.970061 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.970062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.970073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.970149 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:11.970337 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.970919 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.971938 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.972325 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.972415 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.972423 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.972577 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.972616 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:11.972704 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.969969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:41735 (local address 127.1.44.130:37151)
0924 00:14:11.970015 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.970017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.970157 (+ 140us) server_negotiation.cc:407] Connection header received
0924 00:14:11.970186 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.970187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.970198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.970272 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:11.970428 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.971019 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.971981 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.972347 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.972441 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.972450 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.972602 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.972638 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:11.972716 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.969871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36825 (local address 127.1.44.131:38507)
0924 00:14:11.969941 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.969953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.970003 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.970173 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.970174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.970176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:11.970244 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.970246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.971007 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.971008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.971840 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.971843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.972411 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.972412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.972493 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.972496 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.972559 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.972598 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260924 00:14:11.972730 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.969889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37151 (local address 127.1.44.131:41735)
0924 00:14:11.970046 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:11.970055 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:11.970115 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:11.970288 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:11.970289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:11.970291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:11.970349 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.970350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.971095 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.971096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.971888 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:11.971890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.972442 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.972443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:11.972525 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.972528 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:11.972595 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 00:14:11.972631 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260924 00:14:11.973058 2474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5492f5354f9f885c3df168cc00af" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fbd2ee570b460aa078a66344293252" is_pre_election: true
I20260924 00:14:11.973228 2474 raft_consensus.cc:2454] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 0.
I20260924 00:14:11.973253 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5492f5354f9f885c3df168cc00af" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" is_pre_election: true
I20260924 00:14:11.973415 2614 raft_consensus.cc:2454] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 0.
I20260924 00:14:11.973462 2685 leader_election.cc:302] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de6f964b971543f49ced0002b3b44660, e1fbd2ee570b460aa078a66344293252; no voters:
I20260924 00:14:11.973608 2831 raft_consensus.cc:2781] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:11.973665 2831 raft_consensus.cc:482] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:11.973712 2831 raft_consensus.cc:3037] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:11.974431 2831 raft_consensus.cc:504] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.974567 2831 leader_election.cc:288] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 election: Requested vote from peers e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825), 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151)
I20260924 00:14:11.974696 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5492f5354f9f885c3df168cc00af" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5"
I20260924 00:14:11.974701 2474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0c5492f5354f9f885c3df168cc00af" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fbd2ee570b460aa078a66344293252"
I20260924 00:14:11.974776 2474 raft_consensus.cc:3037] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:11.974794 2614 raft_consensus.cc:3037] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:11.975546 2474 raft_consensus.cc:2454] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 1.
I20260924 00:14:11.975565 2614 raft_consensus.cc:2454] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 1.
I20260924 00:14:11.975698 2685 leader_election.cc:302] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de6f964b971543f49ced0002b3b44660, e1fbd2ee570b460aa078a66344293252; no voters:
I20260924 00:14:11.975821 2831 raft_consensus.cc:2781] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:11.975975 2831 raft_consensus.cc:686] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 1 LEADER]: Becoming Leader. State: Replica: de6f964b971543f49ced0002b3b44660, State: Running, Role: LEADER
I20260924 00:14:11.976059 2831 consensus_queue.cc:237] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:11.976836 2316 catalog_manager.cc:5495] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 reported cstate change: term changed from 0 to 1, leader changed from <none> to de6f964b971543f49ced0002b3b44660 (127.1.44.131). New cstate: current_term: 1 leader_uuid: "de6f964b971543f49ced0002b3b44660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } health_report { overall_health: HEALTHY } } }
I20260924 00:14:11.988341 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:11.985648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.1.44.131:36557)
0924 00:14:11.985696 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:11.985698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:11.985741 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:14:11.985815 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:11.985816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:11.985838 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:11.985899 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:11.986048 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.986573 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.987441 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:11.987659 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:11.987717 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:11.987887 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:11.987969 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:11.988082 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:11.988101 (+ 19us) server_negotiation.cc:299] Negotiation successful
0924 00:14:11.988118 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:11.990077 2474 raft_consensus.cc:1261] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Refusing update from remote peer de6f964b971543f49ced0002b3b44660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:11.990353 2614 raft_consensus.cc:1261] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Refusing update from remote peer de6f964b971543f49ced0002b3b44660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:11.990383 2831 consensus_queue.cc:1035] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:11.990619 2831 consensus_queue.cc:1035] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:11.993150 2838 mvcc.cc:205] Tried to move back new op lower bound from 7332695457748639744 to 7332695457694638080. Current Snapshot: MvccSnapshot[applied={T|T < 7332695457748639744}]
I20260924 00:14:11.995740 2316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:11.995834 2316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:11.996526 2316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: Sending DeleteTablet for 3 replicas of tablet 3f0c5492f5354f9f885c3df168cc00af
I20260924 00:14:11.996814 2454 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0c5492f5354f9f885c3df168cc00af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:11 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41404
I20260924 00:14:11.996889 2594 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0c5492f5354f9f885c3df168cc00af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:11 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33726
I20260924 00:14:11.997022 2734 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0c5492f5354f9f885c3df168cc00af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:11 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260924 00:14:11.997049 2845 tablet_replica.cc:330] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: stopping tablet replica
I20260924 00:14:11.997145 2845 raft_consensus.cc:2229] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:11.997257 2845 raft_consensus.cc:2258] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:11.997272 2847 tablet_replica.cc:330] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: stopping tablet replica
I20260924 00:14:11.997426 2316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:11.997566 2316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:11.997691 2845 ts_tablet_manager.cc:1905] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:11.997973 2846 tablet_replica.cc:330] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: stopping tablet replica
I20260924 00:14:11.998071 2846 raft_consensus.cc:2229] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:11.998178 2846 raft_consensus.cc:2258] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:11.999017 2454 tablet_service.cc:1467] Processing CreateTablet for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 (DEFAULT_TABLE table=test-table [id=9da0dc3008cb4276a9a255c5f7bd0fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:11.999023 2594 tablet_service.cc:1467] Processing CreateTablet for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 (DEFAULT_TABLE table=test-table [id=9da0dc3008cb4276a9a255c5f7bd0fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:11.999136 2594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:11.999061 2847 raft_consensus.cc:2229] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:11.999147 2454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:11.999325 2847 raft_consensus.cc:2258] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:12.000291 2824 tablet_bootstrap.cc:492] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Bootstrap starting.
I20260924 00:14:12.000350 2734 tablet_service.cc:1467] Processing CreateTablet for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 (DEFAULT_TABLE table=test-table [id=9da0dc3008cb4276a9a255c5f7bd0fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:12.000444 2734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:12.000444 2823 tablet_bootstrap.cc:492] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Bootstrap starting.
I20260924 00:14:12.000721 2824 tablet_bootstrap.cc:654] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:12.000844 2823 tablet_bootstrap.cc:654] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:12.001497 2823 tablet_bootstrap.cc:492] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: No bootstrap required, opened a new log
I20260924 00:14:12.001487 2845 ts_tablet_manager.cc:1918] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:12.001547 2823 ts_tablet_manager.cc:1397] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:12.001559 2845 log.cc:1196] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/3f0c5492f5354f9f885c3df168cc00af
I20260924 00:14:12.001706 2823 raft_consensus.cc:348] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.001811 2823 raft_consensus.cc:374] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:12.001835 2823 raft_consensus.cc:729] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3c74afaa7047dd9ee98e2a2d846cc5, State: Initialized, Role: FOLLOWER
I20260924 00:14:12.001838 2827 tablet_bootstrap.cc:492] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Bootstrap starting.
I20260924 00:14:12.001848 2845 ts_tablet_manager.cc:1939] T 3f0c5492f5354f9f885c3df168cc00af P e1fbd2ee570b460aa078a66344293252: Deleting consensus metadata
I20260924 00:14:12.001880 2823 consensus_queue.cc:260] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.001981 2823 ts_tablet_manager.cc:1428] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:12.002691 2827 tablet_bootstrap.cc:654] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:12.002667 2301 catalog_manager.cc:4841] TS e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825): tablet 3f0c5492f5354f9f885c3df168cc00af (table test-table [id=e28aec49ceea4cfe9b8d006177d1c80d]) successfully deleted
I20260924 00:14:11.998876 2846 ts_tablet_manager.cc:1905] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:12.003139 2847 ts_tablet_manager.cc:1905] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:12.004161 2846 ts_tablet_manager.cc:1918] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:12.004231 2846 log.cc:1196] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/3f0c5492f5354f9f885c3df168cc00af
I20260924 00:14:12.004496 2846 ts_tablet_manager.cc:1939] T 3f0c5492f5354f9f885c3df168cc00af P 0a3c74afaa7047dd9ee98e2a2d846cc5: Deleting consensus metadata
I20260924 00:14:12.004606 2847 ts_tablet_manager.cc:1918] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:12.004678 2847 log.cc:1196] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/3f0c5492f5354f9f885c3df168cc00af
I20260924 00:14:12.004943 2303 catalog_manager.cc:4841] TS 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151): tablet 3f0c5492f5354f9f885c3df168cc00af (table test-table [id=e28aec49ceea4cfe9b8d006177d1c80d]) successfully deleted
I20260924 00:14:12.004969 2847 ts_tablet_manager.cc:1939] T 3f0c5492f5354f9f885c3df168cc00af P de6f964b971543f49ced0002b3b44660: Deleting consensus metadata
I20260924 00:14:12.005246 2827 tablet_bootstrap.cc:492] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: No bootstrap required, opened a new log
I20260924 00:14:12.005297 2827 ts_tablet_manager.cc:1397] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 00:14:12.005348 2301 catalog_manager.cc:4841] TS de6f964b971543f49ced0002b3b44660 (127.1.44.131:36557): tablet 3f0c5492f5354f9f885c3df168cc00af (table test-table [id=e28aec49ceea4cfe9b8d006177d1c80d]) successfully deleted
I20260924 00:14:12.005445 2827 raft_consensus.cc:348] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.005541 2827 raft_consensus.cc:374] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:12.005569 2827 raft_consensus.cc:729] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6f964b971543f49ced0002b3b44660, State: Initialized, Role: FOLLOWER
I20260924 00:14:12.005625 2827 consensus_queue.cc:260] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.005720 2827 ts_tablet_manager.cc:1428] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:12.005916 2824 tablet_bootstrap.cc:492] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: No bootstrap required, opened a new log
I20260924 00:14:12.005960 2824 ts_tablet_manager.cc:1397] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260924 00:14:12.006099 2824 raft_consensus.cc:348] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.006170 2824 raft_consensus.cc:374] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:12.006188 2824 raft_consensus.cc:729] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fbd2ee570b460aa078a66344293252, State: Initialized, Role: FOLLOWER
I20260924 00:14:12.006225 2824 consensus_queue.cc:260] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.006316 2824 ts_tablet_manager.cc:1428] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:12.045943 2831 raft_consensus.cc:482] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:12.046064 2831 raft_consensus.cc:504] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.046269 2831 leader_election.cc:288] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825), 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151)
I20260924 00:14:12.046453 2474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf8e2b87bd42ebbb437a0a0cbf26b8" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fbd2ee570b460aa078a66344293252" is_pre_election: true
I20260924 00:14:12.046464 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf8e2b87bd42ebbb437a0a0cbf26b8" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" is_pre_election: true
I20260924 00:14:12.046568 2614 raft_consensus.cc:2454] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 0.
I20260924 00:14:12.046578 2474 raft_consensus.cc:2454] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 0.
I20260924 00:14:12.046732 2685 leader_election.cc:302] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de6f964b971543f49ced0002b3b44660, e1fbd2ee570b460aa078a66344293252; no voters:
I20260924 00:14:12.046829 2831 raft_consensus.cc:2781] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:12.046867 2831 raft_consensus.cc:482] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:12.046895 2831 raft_consensus.cc:3037] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:12.047592 2831 raft_consensus.cc:504] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.047724 2831 leader_election.cc:288] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 election: Requested vote from peers e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825), 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151)
I20260924 00:14:12.047873 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf8e2b87bd42ebbb437a0a0cbf26b8" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5"
I20260924 00:14:12.047950 2614 raft_consensus.cc:3037] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:12.047920 2474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbbf8e2b87bd42ebbb437a0a0cbf26b8" candidate_uuid: "de6f964b971543f49ced0002b3b44660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fbd2ee570b460aa078a66344293252"
I20260924 00:14:12.048012 2474 raft_consensus.cc:3037] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:12.048597 2474 raft_consensus.cc:2454] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 1.
I20260924 00:14:12.048674 2614 raft_consensus.cc:2454] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de6f964b971543f49ced0002b3b44660 in term 1.
I20260924 00:14:12.048782 2685 leader_election.cc:302] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de6f964b971543f49ced0002b3b44660, e1fbd2ee570b460aa078a66344293252; no voters:
I20260924 00:14:12.048868 2831 raft_consensus.cc:2781] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:12.048930 2831 raft_consensus.cc:686] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 1 LEADER]: Becoming Leader. State: Replica: de6f964b971543f49ced0002b3b44660, State: Running, Role: LEADER
I20260924 00:14:12.048972 2831 consensus_queue.cc:237] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.049530 2316 catalog_manager.cc:5495] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 reported cstate change: term changed from 0 to 1, leader changed from <none> to de6f964b971543f49ced0002b3b44660 (127.1.44.131). New cstate: current_term: 1 leader_uuid: "de6f964b971543f49ced0002b3b44660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } health_report { overall_health: HEALTHY } } }
W20260924 00:14:12.059679 2521 tablet.cc:2369] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 00:14:12 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:12.091557 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.083997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34137 (local address 127.0.0.1:35442)
0924 00:14:12.084176 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:12.084185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:12.084262 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:12.084448 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:12.084449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:12.084761 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:12.084811 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.084813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.085711 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.085711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.086278 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.086280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.086763 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.086764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.086806 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.087356 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:12.087361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:12.088856 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:12.090570 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:12.090576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:12.090577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:12.090761 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:12.090966 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:12.090966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:12.090966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:12.091020 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:12.091304 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:12.091309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:12.091347 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 00:14:12.091400 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:14:12.091485 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260924 00:14:12.091657 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.084073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.1.44.190:34137)
0924 00:14:12.084136 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.084138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.084246 (+ 108us) server_negotiation.cc:407] Connection header received
0924 00:14:12.084319 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.084320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.084330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.084792 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:12.084873 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.085638 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.086351 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.086678 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.086762 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.088934 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:12.088936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:12.088936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:12.088943 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:12.090487 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:12.090834 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:12.090835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:12.090835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:12.090898 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:12.091080 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:12.091081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:12.091082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:12.091241 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:12.091307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.091449 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.091556 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":35}
I20260924 00:14:12.093495 2316 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:35442:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 00:14:12.093719 2316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:12.095104 2734 tablet_service.cc:1467] Processing CreateTablet for tablet b65bec518a6746a9b576e0dbb5059301 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=574a5641adef48808e458fd9771fd783]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:12.095116 2594 tablet_service.cc:1467] Processing CreateTablet for tablet b65bec518a6746a9b576e0dbb5059301 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=574a5641adef48808e458fd9771fd783]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:12.095280 2594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b65bec518a6746a9b576e0dbb5059301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:12.095288 2734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b65bec518a6746a9b576e0dbb5059301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:12.096699 2827 tablet_bootstrap.cc:492] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: Bootstrap starting.
I20260924 00:14:12.096776 2454 tablet_service.cc:1467] Processing CreateTablet for tablet b65bec518a6746a9b576e0dbb5059301 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=574a5641adef48808e458fd9771fd783]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:12.096901 2454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b65bec518a6746a9b576e0dbb5059301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:12.097561 2827 tablet_bootstrap.cc:654] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:12.097852 2824 tablet_bootstrap.cc:492] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: Bootstrap starting.
I20260924 00:14:12.097895 2823 tablet_bootstrap.cc:492] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Bootstrap starting.
I20260924 00:14:12.098212 2824 tablet_bootstrap.cc:654] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:12.098296 2823 tablet_bootstrap.cc:654] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:12.098753 2824 tablet_bootstrap.cc:492] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: No bootstrap required, opened a new log
I20260924 00:14:12.098788 2824 ts_tablet_manager.cc:1397] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:12.098834 2827 tablet_bootstrap.cc:492] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: No bootstrap required, opened a new log
I20260924 00:14:12.098874 2827 ts_tablet_manager.cc:1397] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:12.098894 2824 raft_consensus.cc:348] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.098940 2824 raft_consensus.cc:374] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:12.098953 2824 raft_consensus.cc:729] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fbd2ee570b460aa078a66344293252, State: Initialized, Role: FOLLOWER
I20260924 00:14:12.099012 2824 consensus_queue.cc:260] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.099004 2827 raft_consensus.cc:348] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.099074 2827 raft_consensus.cc:374] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:12.099097 2827 raft_consensus.cc:729] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6f964b971543f49ced0002b3b44660, State: Initialized, Role: FOLLOWER
I20260924 00:14:12.099123 2824 ts_tablet_manager.cc:1428] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:12.099138 2827 consensus_queue.cc:260] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.099246 2827 ts_tablet_manager.cc:1428] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:12.099256 2831 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.099174 2830 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.099447 2831 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:12.099452 2830 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:12.100210 2823 tablet_bootstrap.cc:492] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: No bootstrap required, opened a new log
I20260924 00:14:12.100265 2823 ts_tablet_manager.cc:1397] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:12.100397 2823 raft_consensus.cc:348] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.100484 2823 raft_consensus.cc:374] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:12.100505 2823 raft_consensus.cc:729] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3c74afaa7047dd9ee98e2a2d846cc5, State: Initialized, Role: FOLLOWER
I20260924 00:14:12.100553 2823 consensus_queue.cc:260] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.100637 2823 ts_tablet_manager.cc:1428] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:12.100788 2829 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.100874 2829 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 00:14:12.131338 2801 tablet.cc:2369] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:12.165971 2829 raft_consensus.cc:482] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:12.166070 2829 raft_consensus.cc:504] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.166440 2829 leader_election.cc:288] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825), de6f964b971543f49ced0002b3b44660 (127.1.44.131:36557)
I20260924 00:14:12.169500 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.166700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:49735 (local address 127.1.44.129:36825)
0924 00:14:12.166755 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.166757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.166767 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:12.166881 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.166883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.166897 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.166960 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:12.167142 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.167682 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.168643 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.169209 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.169263 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.169270 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.169422 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.169450 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 00:14:12.169528 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.166617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36825 (local address 127.1.44.130:49735)
0924 00:14:12.166683 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:12.166698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:12.166762 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:12.166981 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:12.166983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:12.166985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:12.167063 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.167065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.167752 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.167754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.168549 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.168551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.169276 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.169277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.169347 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.169350 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:12.169406 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 00:14:12.169439 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 00:14:12.169793 2474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65bec518a6746a9b576e0dbb5059301" candidate_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fbd2ee570b460aa078a66344293252" is_pre_election: true
I20260924 00:14:12.169883 2474 raft_consensus.cc:2454] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3c74afaa7047dd9ee98e2a2d846cc5 in term 0.
I20260924 00:14:12.170077 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.166795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48731 (local address 127.1.44.131:36557)
0924 00:14:12.166841 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.166844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.166990 (+ 146us) server_negotiation.cc:407] Connection header received
0924 00:14:12.167018 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.167019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.167030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.167083 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:12.167305 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.167770 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.169165 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.169699 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.169761 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.169769 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.169960 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.170004 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:12.170077 2545 leader_election.cc:302] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3c74afaa7047dd9ee98e2a2d846cc5, e1fbd2ee570b460aa078a66344293252; no voters:
I20260924 00:14:12.170221 2829 raft_consensus.cc:2781] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:12.170279 2829 raft_consensus.cc:482] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:12.170310 2829 raft_consensus.cc:3037] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:12.170385 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.166733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:36557 (local address 127.1.44.130:48731)
0924 00:14:12.166890 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:12.166900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:12.166949 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:12.167118 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:12.167119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:12.167121 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:12.167198 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.167200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.167851 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.167851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.169076 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.169078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.169814 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.169815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.169877 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.169880 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:12.170176 (+ 296us) client_negotiation.cc:240] Negotiation successful
0924 00:14:12.170223 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260924 00:14:12.170646 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65bec518a6746a9b576e0dbb5059301" candidate_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de6f964b971543f49ced0002b3b44660" is_pre_election: true
I20260924 00:14:12.170794 2754 raft_consensus.cc:2454] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3c74afaa7047dd9ee98e2a2d846cc5 in term 0.
I20260924 00:14:12.171168 2829 raft_consensus.cc:504] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.171314 2829 leader_election.cc:288] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [CANDIDATE]: Term 1 election: Requested vote from peers e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825), de6f964b971543f49ced0002b3b44660 (127.1.44.131:36557)
I20260924 00:14:12.171455 2474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65bec518a6746a9b576e0dbb5059301" candidate_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1fbd2ee570b460aa078a66344293252"
I20260924 00:14:12.171492 2754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b65bec518a6746a9b576e0dbb5059301" candidate_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de6f964b971543f49ced0002b3b44660"
I20260924 00:14:12.171523 2474 raft_consensus.cc:3037] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:12.171571 2754 raft_consensus.cc:3037] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:12.172135 2474 raft_consensus.cc:2454] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3c74afaa7047dd9ee98e2a2d846cc5 in term 1.
I20260924 00:14:12.172236 2754 raft_consensus.cc:2454] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3c74afaa7047dd9ee98e2a2d846cc5 in term 1.
I20260924 00:14:12.172283 2545 leader_election.cc:302] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3c74afaa7047dd9ee98e2a2d846cc5, e1fbd2ee570b460aa078a66344293252; no voters:
I20260924 00:14:12.172367 2829 raft_consensus.cc:2781] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:12.172549 2829 raft_consensus.cc:686] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3c74afaa7047dd9ee98e2a2d846cc5, State: Running, Role: LEADER
I20260924 00:14:12.172628 2829 consensus_queue.cc:237] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } }
I20260924 00:14:12.172966 2862 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: TxnStatusTablet state changed. Reason: New leader 0a3c74afaa7047dd9ee98e2a2d846cc5. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.173059 2862 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:12.173228 2864 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 00:14:12.173331 2864 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 00:14:12.173295 2316 catalog_manager.cc:5495] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130). New cstate: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:12.193410 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.190883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37151 (local address 127.0.0.1:33740)
0924 00:14:12.190955 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:12.190967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:12.191018 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:12.191247 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:12.191248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:12.191249 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:12.191308 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.191310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.191946 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.191947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.192521 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.192522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.192888 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.192889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.192953 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.192968 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:12.193239 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:12.193248 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 00:14:12.193298 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:14:12.193343 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 00:14:12.193398 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.190952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.1.44.130:37151)
0924 00:14:12.191006 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.191008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.191018 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:12.191117 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.191118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.191135 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.191235 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:12.191385 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.191861 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.192612 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.192817 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.192890 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.193065 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:12.193171 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:12.193252 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.193302 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.193321 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 00:14:12.194530 2474 raft_consensus.cc:1261] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Refusing update from remote peer 0a3c74afaa7047dd9ee98e2a2d846cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:12.194551 2754 raft_consensus.cc:1261] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Refusing update from remote peer 0a3c74afaa7047dd9ee98e2a2d846cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:12.194792 2862 consensus_queue.cc:1035] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:12.194908 2829 consensus_queue.cc:1035] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:12.195457 2830 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: TxnStatusTablet state changed. Reason: New leader 0a3c74afaa7047dd9ee98e2a2d846cc5. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.195547 2830 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:12.195631 2831 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: TxnStatusTablet state changed. Reason: New leader 0a3c74afaa7047dd9ee98e2a2d846cc5. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.195725 2831 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:12.196251 2862 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.196339 2862 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:12.196625 2831 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.196712 2831 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P de6f964b971543f49ced0002b3b44660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:12.197275 2829 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.197357 2829 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P 0a3c74afaa7047dd9ee98e2a2d846cc5: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:12.197468 2837 mvcc.cc:205] Tried to move back new op lower bound from 7332695458586521600 to 7332695458499588096. Current Snapshot: MvccSnapshot[applied={T|T < 7332695458586521600}]
I20260924 00:14:12.197593 2830 tablet_replica.cc:439] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 } } peers { permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 } } peers { permanent_uuid: "de6f964b971543f49ced0002b3b44660" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 36557 } } }
I20260924 00:14:12.197667 2830 tablet_replica.cc:442] T b65bec518a6746a9b576e0dbb5059301 P e1fbd2ee570b460aa078a66344293252: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:12.202080 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.199875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:55699 (local address 127.1.44.130:37151)
0924 00:14:12.199908 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.199910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.199934 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:14:12.200006 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.200007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.200015 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.200053 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:12.200201 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.200659 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.201397 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.201600 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.201683 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.201849 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:12.201885 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:12.201944 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.202003 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.202020 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:12.202126 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.199808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37151 (local address 127.1.44.131:55699)
0924 00:14:12.199882 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:12.199894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:12.199950 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:12.200064 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:12.200064 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:12.200066 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:12.200132 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.200134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.200750 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.200751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.201318 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.201320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.201686 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.201687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.201742 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.201757 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:12.201945 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:12.201947 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:12.201999 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 00:14:12.202044 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 00:14:12.204505 2614 raft_consensus.cc:1261] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Refusing update from remote peer de6f964b971543f49ced0002b3b44660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:12.204627 2474 raft_consensus.cc:1261] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Refusing update from remote peer de6f964b971543f49ced0002b3b44660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:12.204803 2831 consensus_queue.cc:1035] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1fbd2ee570b460aa078a66344293252" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:12.204905 2833 consensus_queue.cc:1035] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3c74afaa7047dd9ee98e2a2d846cc5" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:12.205957 2836 mvcc.cc:205] Tried to move back new op lower bound from 7332695458628272128 to 7332695457993007104. Current Snapshot: MvccSnapshot[applied={T|T < 7332695458628272128}]
I20260924 00:14:12.213017 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.210430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48873 (local address 127.1.44.131:36557)
0924 00:14:12.210469 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.210471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.210479 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:12.210499 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.210500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.210509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.210570 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:12.210660 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.211398 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.212281 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.212536 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.212624 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.212804 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:12.212836 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:12.212877 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.212931 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.212950 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:12.213029 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.210340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:36557 (local address 127.1.44.130:48873)
0924 00:14:12.210393 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:12.210404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:12.210443 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:12.210571 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:12.210571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:12.210577 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:12.210622 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.210623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.211479 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.211480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.212186 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:12.212188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.212636 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.212637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:12.212700 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.212716 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:12.212888 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:12.212889 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:12.212930 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 00:14:12.212966 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 00:14:12.221836 2317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:12.221938 2317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:12.222517 2317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80b1c66a4aad4b8cbfd40d3db8b3f37f: Sending DeleteTablet for 3 replicas of tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8
I20260924 00:14:12.222759 2594 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:12 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33726
I20260924 00:14:12.222841 2846 tablet_replica.cc:330] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: stopping tablet replica
I20260924 00:14:12.222842 2454 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:12 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41404
I20260924 00:14:12.222878 2846 raft_consensus.cc:2229] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:12.222919 2846 raft_consensus.cc:2258] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:12.222946 2845 tablet_replica.cc:330] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: stopping tablet replica
I20260924 00:14:12.222935 2734 tablet_service.cc:1514] Processing DeleteTablet for tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:12 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260924 00:14:12.222991 2845 raft_consensus.cc:2229] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:12.223011 2847 tablet_replica.cc:330] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: stopping tablet replica
I20260924 00:14:12.223037 2845 raft_consensus.cc:2258] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:12.223054 2847 raft_consensus.cc:2229] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:12.223112 2847 raft_consensus.cc:2258] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:12.223376 2845 ts_tablet_manager.cc:1905] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:12.223519 2847 ts_tablet_manager.cc:1905] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:12.223572 2846 ts_tablet_manager.cc:1905] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:12.225219 2847 ts_tablet_manager.cc:1918] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:12.225313 2847 log.cc:1196] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/bbbf8e2b87bd42ebbb437a0a0cbf26b8
I20260924 00:14:12.225601 2847 ts_tablet_manager.cc:1939] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P de6f964b971543f49ced0002b3b44660: Deleting consensus metadata
I20260924 00:14:12.225898 2846 ts_tablet_manager.cc:1918] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:12.225922 2301 catalog_manager.cc:4841] TS de6f964b971543f49ced0002b3b44660 (127.1.44.131:36557): tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 (table test-table [id=9da0dc3008cb4276a9a255c5f7bd0fd6]) successfully deleted
I20260924 00:14:12.225956 2846 log.cc:1196] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/bbbf8e2b87bd42ebbb437a0a0cbf26b8
I20260924 00:14:12.226212 2846 ts_tablet_manager.cc:1939] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P 0a3c74afaa7047dd9ee98e2a2d846cc5: Deleting consensus metadata
I20260924 00:14:12.226493 2845 ts_tablet_manager.cc:1918] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:12.226519 2303 catalog_manager.cc:4841] TS 0a3c74afaa7047dd9ee98e2a2d846cc5 (127.1.44.130:37151): tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 (table test-table [id=9da0dc3008cb4276a9a255c5f7bd0fd6]) successfully deleted
I20260924 00:14:12.226552 2845 log.cc:1196] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/bbbf8e2b87bd42ebbb437a0a0cbf26b8
I20260924 00:14:12.226733 2845 ts_tablet_manager.cc:1939] T bbbf8e2b87bd42ebbb437a0a0cbf26b8 P e1fbd2ee570b460aa078a66344293252: Deleting consensus metadata
I20260924 00:14:12.227275 2301 catalog_manager.cc:4841] TS e1fbd2ee570b460aa078a66344293252 (127.1.44.129:36825): tablet bbbf8e2b87bd42ebbb437a0a0cbf26b8 (table test-table [id=9da0dc3008cb4276a9a255c5f7bd0fd6]) successfully deleted
Sep 24 00:14:12 dist-test-slave-cm7b krb5kdc[2272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208851, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:12.234500 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.225145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.1.44.129:36825)
0924 00:14:12.225186 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.225188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.225408 (+ 220us) server_negotiation.cc:407] Connection header received
0924 00:14:12.225514 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.225515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.225525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.225589 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:12.226355 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.227085 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.228399 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.228696 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.228769 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.231724 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:12.231733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:12.231733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:12.231741 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:12.233356 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:12.233741 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:12.233744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:12.233744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:12.233810 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:12.234030 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:12.234031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:12.234031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:12.234154 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:12.234269 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.234389 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.234444 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 00:14:12.234977 2498 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41406
I20260924 00:14:12.241554 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:12.235966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.1.44.190:34137)
0924 00:14:12.236009 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:12.236011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:12.236067 (+ 56us) server_negotiation.cc:407] Connection header received
0924 00:14:12.236137 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:12.236139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:12.236148 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:12.236202 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:12.236653 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.237219 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.237897 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:12.238106 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:12.238182 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:12.239278 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:12.239279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:12.239279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:12.239286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:12.240499 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:12.240817 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:12.240818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:12.240818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:12.240869 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:12.241074 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:12.241074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:12.241075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:12.241183 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:12.241280 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:12.241388 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 00:14:12.241459 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 00:14:12.241741 2317 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35456
I20260924 00:14:12.242393 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 2389
I20260924 00:14:12.247466 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 2529
I20260924 00:14:12.251973 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 2669
I20260924 00:14:12.257144 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 2285
2026-09-24T00:14:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2146 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:12 dist-test-slave-cm7b krb5kdc[2883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:12 dist-test-slave-cm7b krb5kdc[2883](info): set up 2 sockets
Sep 24 00:14:12 dist-test-slave-cm7b krb5kdc[2883](info): commencing operation
krb5kdc: starting...
W20260924 00:14:13.298839 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:13.415181 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34719
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:13.494855 2896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:13.495057 2896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:13.495086 2896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:13.496268 2896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:13.496310 2896 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 00:14:13.496323 2896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:13.496336 2896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:13.496348 2896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:13.496403 2896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:13.497845 2896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34719
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:13.498106 2896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:13.498359 2896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:13.503084 2901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:13.503090 2904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:13.503109 2902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:13.503255 2896 server_base.cc:1034] running on GCE node
I20260924 00:14:13.503486 2896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:13.503793 2896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:13.504962 2896 hybrid_clock.cc:648] HybridClock initialized: now 1790208853504938 us; error 37 us; skew 500 ppm
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:13.507463 2896 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:13.507911 2896 webserver.cc:460] Webserver started at http://127.1.44.190:35239/ using document root <none> and password file <none>
I20260924 00:14:13.508155 2896 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:13.508200 2896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:13.508369 2896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:13.509881 2896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "3d0cea0f37ba45078ecdbe73ef586394"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "43f0de86c53fd865e1f91d90daf8c055"
server_key_iv: "f1bff652aaf09363370643ed3ba1b064"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.510313 2896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "3d0cea0f37ba45078ecdbe73ef586394"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "43f0de86c53fd865e1f91d90daf8c055"
server_key_iv: "f1bff652aaf09363370643ed3ba1b064"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.512042 2896 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260924 00:14:13.513012 2911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:13.513195 2896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:13.513278 2896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "3d0cea0f37ba45078ecdbe73ef586394"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "43f0de86c53fd865e1f91d90daf8c055"
server_key_iv: "f1bff652aaf09363370643ed3ba1b064"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.513368 2896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:13.527139 2896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:13.528954 2896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:13.529143 2896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:13.533671 2896 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34719
I20260924 00:14:13.533706 2973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34719 every 8 connection(s)
I20260924 00:14:13.534017 2896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:13.534582 2974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:13.537388 2974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Bootstrap starting.
I20260924 00:14:13.537945 2974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:13.538242 2974 log.cc:824] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:13.538913 2974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: No bootstrap required, opened a new log
I20260924 00:14:13.540030 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 2896
I20260924 00:14:13.540109 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:13.540272 1202 external_mini_cluster.cc:1428] Setting key 69daf4acef15f24fcbd337baf0d2ea7f
I20260924 00:14:13.541426 2974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cea0f37ba45078ecdbe73ef586394" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34719 } }
I20260924 00:14:13.541595 2974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:13.541630 2974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0cea0f37ba45078ecdbe73ef586394, State: Initialized, Role: FOLLOWER
I20260924 00:14:13.541754 2974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cea0f37ba45078ecdbe73ef586394" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34719 } }
I20260924 00:14:13.541833 2974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:13.541855 2974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:13.541924 2974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:13.542676 2974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cea0f37ba45078ecdbe73ef586394" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34719 } }
I20260924 00:14:13.542795 2974 leader_election.cc:302] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d0cea0f37ba45078ecdbe73ef586394; no voters:
I20260924 00:14:13.542964 2974 leader_election.cc:288] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:13.543004 2979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:13.543126 2979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0cea0f37ba45078ecdbe73ef586394, State: Running, Role: LEADER
I20260924 00:14:13.543246 2974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:13.543226 2979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cea0f37ba45078ecdbe73ef586394" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34719 } }
I20260924 00:14:13.543541 2981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0cea0f37ba45078ecdbe73ef586394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cea0f37ba45078ecdbe73ef586394" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34719 } } }
I20260924 00:14:13.543635 2981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:13.543537 2980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0cea0f37ba45078ecdbe73ef586394. Latest consensus state: current_term: 1 leader_uuid: "3d0cea0f37ba45078ecdbe73ef586394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0cea0f37ba45078ecdbe73ef586394" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34719 } } }
I20260924 00:14:13.543813 2980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:13.543916 2983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:13.544428 2983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:13.546561 2983 catalog_manager.cc:1347] Generated new cluster ID: c870f25991d64c17971f51948852f04f
I20260924 00:14:13.546613 2983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:13.551258 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.540796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.1.44.190:34719)
0924 00:14:13.541028 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:13.541039 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:13.541079 (+ 40us) server_negotiation.cc:407] Connection header received
0924 00:14:13.541386 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:13.541394 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:13.541465 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:13.541613 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:13.542446 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.543851 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.544940 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.545366 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.545445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.548041 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:13.548044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:13.548045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:13.548057 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:13.549739 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.550115 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.550118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.550118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.550219 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.550428 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.550428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.550429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.550658 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:13.550825 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:13.550917 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 00:14:13.551055 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":107,"threads_started":1}
I20260924 00:14:13.553013 2983 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:13.553655 2983 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:13.564486 2983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Generated new TSK 0
I20260924 00:14:13.564667 2983 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:13.593756 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34719
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:13.693862 3000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:13.694056 3000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:13.694079 3000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:13.695268 3000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:13.695345 3000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:13.695387 3000 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 00:14:13.695411 3000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:13.695433 3000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:13.697955 3000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:13.698212 3000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:13.698454 3000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:13.704780 3006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:13.704789 3008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:13.704897 3000 server_base.cc:1034] running on GCE node
W20260924 00:14:13.705122 3005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:13.705348 3000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:13.705605 3000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:13.706772 3000 hybrid_clock.cc:648] HybridClock initialized: now 1790208853706756 us; error 26 us; skew 500 ppm
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:13.709373 3000 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:13.709740 3000 webserver.cc:460] Webserver started at http://127.1.44.129:37419/ using document root <none> and password file <none>
I20260924 00:14:13.709909 3000 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:13.709947 3000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:13.710031 3000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:13.710999 3000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "b1ce649fabd14a6ba0799c40347487d7"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "c0768058361379b943a32027a1fd3055"
server_key_iv: "7a42537c9653be077c2f789665b7b6bb"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.711371 3000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "b1ce649fabd14a6ba0799c40347487d7"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "c0768058361379b943a32027a1fd3055"
server_key_iv: "7a42537c9653be077c2f789665b7b6bb"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.712587 3000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:13.713236 3015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:13.713387 3000 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:13.713472 3000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "b1ce649fabd14a6ba0799c40347487d7"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "c0768058361379b943a32027a1fd3055"
server_key_iv: "7a42537c9653be077c2f789665b7b6bb"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.713522 3000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:13.728199 3000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:13.730316 3000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:13.730490 3000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:13.731009 3000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:13.731081 3000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:13.731124 3000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:13.731149 3000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:13.737874 3000 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:46323
I20260924 00:14:13.737944 3130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:46323 every 8 connection(s)
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:13.738266 3000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:13.738888 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3000
I20260924 00:14:13.738991 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:13.739171 1202 external_mini_cluster.cc:1428] Setting key ea5caa721c39539369890a0d8bd71a7f
I20260924 00:14:13.741883 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.731525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:40107 (local address 127.1.44.190:34719)
0924 00:14:13.731603 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:13.731606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:13.731808 (+ 202us) server_negotiation.cc:407] Connection header received
0924 00:14:13.732320 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:13.732321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:13.732332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:13.732383 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:13.733313 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.734149 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.735779 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.736123 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.736230 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.738876 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:13.738878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:13.738878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:13.738886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:13.740394 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.740875 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.740876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.740876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.740926 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.741239 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.741240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.741240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.741368 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:13.741444 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:13.741710 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 00:14:13.741793 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 00:14:13.742080 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.731497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.1.44.129:40107)
0924 00:14:13.731699 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:13.731738 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:13.732215 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:13.732469 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:13.732472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:13.732816 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:13.733188 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.733196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.734268 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.734269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.735681 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.735684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.736217 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.736218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.736310 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.736795 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:13.736828 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:13.738750 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:13.740476 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.740482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.740484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.740800 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.740988 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.740989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.740990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.741137 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.741447 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:13.741454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:13.741581 (+ 127us) client_negotiation.cc:769] Sending connection context
0924 00:14:13.741688 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 00:14:13.741849 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":124,"threads_started":1}
I20260924 00:14:13.742511 3131 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34719
I20260924 00:14:13.742596 3131 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:13.742798 3131 heartbeater.cc:511] Master 127.1.44.190:34719 requested a full tablet report, sending...
I20260924 00:14:13.743346 2927 ts_manager.cc:194] Registered new tserver with Master: b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323)
I20260924 00:14:13.744150 2927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:40107
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
I20260924 00:14:13.750504 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.744538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:35723 (local address 127.1.44.190:34719)
0924 00:14:13.744575 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:13.744576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:13.745120 (+ 544us) server_negotiation.cc:407] Connection header received
0924 00:14:13.745148 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:13.745149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:13.745161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:13.745226 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:13.745507 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.746263 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.746909 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.747263 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.747341 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.748185 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:13.748186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:13.748186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:13.748192 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:13.749554 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.749838 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.749838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.749839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.749883 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.750048 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.750049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.750049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.750174 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:13.750237 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:13.750369 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 00:14:13.750448 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:13.750536 3138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.744492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.1.44.129:35723)
0924 00:14:13.745035 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:13.745046 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:13.745109 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:13.745244 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:13.745245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:13.745390 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:13.745438 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.745439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.746359 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.746359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.746830 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.746831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.747337 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.747338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.747394 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.747832 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:13.747835 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:13.748105 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:13.749621 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.749621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.749622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.749774 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.749923 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.749923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.749924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.750001 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.750232 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:13.750233 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:13.750275 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 00:14:13.750328 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:14:13.750419 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":497,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:13.765443 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34719
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:13.843806 3140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:13.843993 3140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:13.844053 3140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:13.845211 3140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:13.845247 3140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:13.845268 3140 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 00:14:13.845279 3140 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:13.845295 3140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:13.846637 3140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:13.846808 3140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:13.846977 3140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:13.851653 3148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:13.851660 3146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:13.851656 3145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:13.851943 3140 server_base.cc:1034] running on GCE node
I20260924 00:14:13.852098 3140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:13.852330 3140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:13.853461 3140 hybrid_clock.cc:648] HybridClock initialized: now 1790208853853454 us; error 29 us; skew 500 ppm
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:13.855906 3140 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:13.856364 3140 webserver.cc:460] Webserver started at http://127.1.44.130:46799/ using document root <none> and password file <none>
I20260924 00:14:13.856590 3140 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:13.856654 3140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:13.856791 3140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:13.857805 3140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "325c935b654d43c5b7a73a8ab4bf0bf6"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "060bc3050dae51927fc0f35779a37508"
server_key_iv: "8d1fbec1096721c36e7d87aa3c3c4b02"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.858161 3140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "325c935b654d43c5b7a73a8ab4bf0bf6"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "060bc3050dae51927fc0f35779a37508"
server_key_iv: "8d1fbec1096721c36e7d87aa3c3c4b02"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.859481 3140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:13.860195 3155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:13.860369 3140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:13.860466 3140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "325c935b654d43c5b7a73a8ab4bf0bf6"
format_stamp: "Formatted at 2026-09-24 00:14:13 on dist-test-slave-cm7b"
server_key: "060bc3050dae51927fc0f35779a37508"
server_key_iv: "8d1fbec1096721c36e7d87aa3c3c4b02"
server_key_version: "encryptionkey@0"
I20260924 00:14:13.860548 3140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:13.875851 3140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:13.878063 3140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:13.878227 3140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:13.878805 3140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:13.878854 3140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:13.878885 3140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:13.878904 3140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:14:13 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:13.885926 3140 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:39951
I20260924 00:14:13.886049 3270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:39951 every 8 connection(s)
I20260924 00:14:13.886298 3140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:13.888733 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.879248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:54035 (local address 127.1.44.190:34719)
0924 00:14:13.879294 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:13.879297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:13.879565 (+ 268us) server_negotiation.cc:407] Connection header received
0924 00:14:13.880039 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:13.880040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:13.880052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:13.880103 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:13.880858 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.881672 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.883403 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.883673 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.883760 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.885931 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:13.885932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:13.885932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:13.885940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:13.887422 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.887820 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.887822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.887822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.887879 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.888121 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.888122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.888123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.888239 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:13.888314 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:13.888567 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 00:14:13.888641 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:14:13.888922 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.879255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.1.44.130:54035)
0924 00:14:13.879426 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:13.879464 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:13.879931 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:13.880173 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:13.880176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:13.880523 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:13.880780 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.880785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.881769 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.881771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.883309 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.883312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.883775 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.883776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.883902 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.884375 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:13.884383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:13.885832 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:13.887511 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.887516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.887518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.887732 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.887944 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.887945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.887945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.888055 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.888304 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:13.888310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:13.888442 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 00:14:13.888547 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:14:13.888702 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260924 00:14:13.889385 3271 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34719
I20260924 00:14:13.889469 3271 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:13.889652 3271 heartbeater.cc:511] Master 127.1.44.190:34719 requested a full tablet report, sending...
I20260924 00:14:13.889761 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3140
I20260924 00:14:13.889849 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:13.890035 1202 external_mini_cluster.cc:1428] Setting key 2c21e92f27847bb855ead97d53895f22
I20260924 00:14:13.890417 2927 ts_manager.cc:194] Registered new tserver with Master: 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951)
I20260924 00:14:13.890913 2927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:54035
I20260924 00:14:13.896035 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.890374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:60007 (local address 127.1.44.190:34719)
0924 00:14:13.890408 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:13.890410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:13.890518 (+ 108us) server_negotiation.cc:407] Connection header received
0924 00:14:13.890611 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:13.890611 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:13.890619 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:13.890665 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:13.890958 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.891692 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.892361 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.892605 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.892659 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.893618 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:13.893619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:13.893619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:13.893624 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:13.894908 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.895323 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.895324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.895324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.895370 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:13.895576 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:13.895577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:13.895577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:13.895696 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:13.895749 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:13.895926 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 00:14:13.895981 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:13.896275 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:13.890331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.1.44.130:60007)
0924 00:14:13.890463 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:13.890474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:13.890548 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:13.890669 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:13.890669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:13.890836 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:13.890894 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.890895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.891790 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.891791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.892252 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:13.892253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:13.892677 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:13.892678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:13.892758 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:13.893121 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:13.893126 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:13.893528 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:13.894989 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.894990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.894991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.895246 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.895438 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:13.895439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:13.895439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:13.895508 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:13.895764 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:13.895765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:13.895814 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 00:14:13.895902 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 00:14:13.896015 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:13.915518 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34719
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:13.995108 3280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:13.995361 3280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:13.995388 3280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:13.996470 3280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:13.996506 3280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:13.996526 3280 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 00:14:13.996538 3280 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:13.996555 3280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:13.997913 3280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:13.998097 3280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:13.998265 3280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:14.003180 3285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:14.003398 3286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:14.003386 3288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:14.003381 3280 server_base.cc:1034] running on GCE node
I20260924 00:14:14.003755 3280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:14.003993 3280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:14.005151 3280 hybrid_clock.cc:648] HybridClock initialized: now 1790208854005148 us; error 36 us; skew 500 ppm
Sep 24 00:14:14 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208854, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:14.007547 3280 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:14.007969 3280 webserver.cc:460] Webserver started at http://127.1.44.131:36415/ using document root <none> and password file <none>
I20260924 00:14:14.008193 3280 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:14.008261 3280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:14.008410 3280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:14.009459 3280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "cacc15bff4d54b79828fd42286619e46"
format_stamp: "Formatted at 2026-09-24 00:14:14 on dist-test-slave-cm7b"
server_key: "d008cae8245cef3a6d94923528fa4e7c"
server_key_iv: "4c3bc461e678b81a4c538de568ad08b1"
server_key_version: "encryptionkey@0"
I20260924 00:14:14.009824 3280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "cacc15bff4d54b79828fd42286619e46"
format_stamp: "Formatted at 2026-09-24 00:14:14 on dist-test-slave-cm7b"
server_key: "d008cae8245cef3a6d94923528fa4e7c"
server_key_iv: "4c3bc461e678b81a4c538de568ad08b1"
server_key_version: "encryptionkey@0"
I20260924 00:14:14.011101 3280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:14.011837 3295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.012002 3280 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:14.012099 3280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "cacc15bff4d54b79828fd42286619e46"
format_stamp: "Formatted at 2026-09-24 00:14:14 on dist-test-slave-cm7b"
server_key: "d008cae8245cef3a6d94923528fa4e7c"
server_key_iv: "4c3bc461e678b81a4c538de568ad08b1"
server_key_version: "encryptionkey@0"
I20260924 00:14:14.012179 3280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:14.028973 3280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:14.031281 3280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:14.031435 3280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:14.031916 3280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:14.031975 3280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.032020 3280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:14.032049 3280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:14:14 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208854, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:14.039364 3280 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:41425
I20260924 00:14:14.039464 3410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:41425 every 8 connection(s)
I20260924 00:14:14.039767 3280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:14.040032 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3280
I20260924 00:14:14.040161 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:14.040365 1202 external_mini_cluster.cc:1428] Setting key fa22e0c20e76c51047beb81f02d06456
I20260924 00:14:14.042122 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.032456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:45243 (local address 127.1.44.190:34719)
0924 00:14:14.032507 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.032509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.032770 (+ 261us) server_negotiation.cc:407] Connection header received
0924 00:14:14.033353 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.033355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.033368 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.033442 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:14.034370 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.035222 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.036336 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.036583 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.036671 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.038864 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:14.038866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:14.038866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:14.038875 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:14.040725 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.041116 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.041118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.041119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.041172 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.041450 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.041451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.041451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.041564 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:14.041637 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.041954 (+ 317us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.042036 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:14.042287 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.032461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.1.44.131:45243)
0924 00:14:14.032648 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.032684 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.033199 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.033563 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.033568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.033946 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:14.034263 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.034270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.035323 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.035324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.036235 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.036237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.036696 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.036697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.036821 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.037273 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:14.037281 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:14.038791 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:14.040808 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:14.040811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:14.040813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:14.041025 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:14.041260 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:14.041261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:14.041261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:14.041375 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:14.041630 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:14.041635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:14.041796 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.041921 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.042070 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260924 00:14:14.042750 3411 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34719
I20260924 00:14:14.042842 3411 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:14.043042 3411 heartbeater.cc:511] Master 127.1.44.190:34719 requested a full tablet report, sending...
I20260924 00:14:14.043418 2928 ts_manager.cc:194] Registered new tserver with Master: cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:14.043982 2928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:45243
I20260924 00:14:14.049055 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.043887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:43321 (local address 127.1.44.190:34719)
0924 00:14:14.043962 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.043963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.044118 (+ 155us) server_negotiation.cc:407] Connection header received
0924 00:14:14.044158 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.044159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.044168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.044224 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:14.044606 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.045179 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.045873 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.046039 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.046086 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.046930 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:14.046931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:14.046931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:14.046936 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:14.048172 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.048474 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.048475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.048475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.048518 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.048721 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.048721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.048722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.048799 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:14.048841 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.048942 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.048990 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260924 00:14:14.049142 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.043848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.1.44.131:43321)
0924 00:14:14.043984 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.043995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.044092 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.044230 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.044231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.044427 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:14.044487 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.044489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.045310 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.045311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.045804 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.045805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.046105 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.046106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.046162 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.046598 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:14.046603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:14.046863 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:14.048259 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:14.048260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:14.048261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:14.048416 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:14.048597 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:14.048598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:14.048598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:14.048660 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:14.048848 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:14.048849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:14.048882 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.048928 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.049003 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
I20260924 00:14:14.051892 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:14:14 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:14:14 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:14.073115 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.066646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.1.44.190:34719)
0924 00:14:14.066717 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.066719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.066751 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:14:14.066778 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.066778 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.066786 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.066825 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:14.067402 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.067861 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.068630 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.068807 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.068859 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.070879 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:14.070881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:14.070881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:14.070887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:14.072252 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.072542 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.072542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.072543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.072587 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.072743 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.072744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.072744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.072832 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:14.072904 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.072982 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.073032 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:14.074556 2928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:14.076125 2928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:14.082295 3428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.078259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.0.0.1:45924)
0924 00:14:14.078576 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.078586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.078650 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.078876 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.078877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.078879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.078937 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.078939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.079916 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.079918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.080906 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.080908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.082029 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.082030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.082118 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.082139 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.082205 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.082241 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":62,"threads_started":1}
I20260924 00:14:14.082391 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.078378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.1.44.130:39951)
0924 00:14:14.078556 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.078561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.078642 (+ 81us) server_negotiation.cc:407] Connection header received
0924 00:14:14.078725 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.078728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.078742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.078809 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.079005 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.079836 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.081249 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.081937 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.082030 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.082084 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.082243 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.082286 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260924 00:14:14.082760 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.078586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.1.44.131:41425)
0924 00:14:14.079074 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.079079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.079088 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:14.079124 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.079126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.079140 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.079246 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.079372 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.080407 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.081551 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.082258 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.082326 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.082376 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.082628 (+ 252us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.082667 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":92,"threads_started":1}
I20260924 00:14:14.082749 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.078465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41425 (local address 127.0.0.1:60048)
0924 00:14:14.078753 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.078763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.078805 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.079271 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.079271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.079274 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.079308 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.079310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.080501 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.080502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.081464 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.081466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.082347 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.082348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.082556 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.082560 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.082615 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.082655 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260924 00:14:14.083047 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.078340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.1.44.129:46323)
0924 00:14:14.078634 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.078637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.078647 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:14.078685 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.078688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.078704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.078785 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.078949 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.080400 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.081457 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.081936 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.082027 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.082084 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.082914 (+ 830us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.082956 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":87,"threads_started":1}
I20260924 00:14:14.082748 3429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.078257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:46323 (local address 127.0.0.1:38694)
0924 00:14:14.078424 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.078465 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.078529 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.078795 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.078798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.078801 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.078874 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.078876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.080492 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.080493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.081372 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.081374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.082513 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.082514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.082562 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.082565 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.082618 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.082652 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":33,"thread_start_us":72,"threads_started":1}
I20260924 00:14:14.083113 3205 tablet_service.cc:1467] Processing CreateTablet for tablet 6b0796b6d29842e5a16ee7e3b600f570 (DEFAULT_TABLE table=test-table [id=2097a306df76465d9127617a190fdb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:14.083519 3205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b0796b6d29842e5a16ee7e3b600f570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.083635 3345 tablet_service.cc:1467] Processing CreateTablet for tablet 6b0796b6d29842e5a16ee7e3b600f570 (DEFAULT_TABLE table=test-table [id=2097a306df76465d9127617a190fdb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:14.083932 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b0796b6d29842e5a16ee7e3b600f570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.083845 3065 tablet_service.cc:1467] Processing CreateTablet for tablet 6b0796b6d29842e5a16ee7e3b600f570 (DEFAULT_TABLE table=test-table [id=2097a306df76465d9127617a190fdb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:14.084139 3065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b0796b6d29842e5a16ee7e3b600f570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.086248 3434 tablet_bootstrap.cc:492] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Bootstrap starting.
I20260924 00:14:14.087136 3434 tablet_bootstrap.cc:654] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.087297 3435 tablet_bootstrap.cc:492] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Bootstrap starting.
I20260924 00:14:14.087554 3434 log.cc:824] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:14.087729 3436 tablet_bootstrap.cc:492] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Bootstrap starting.
I20260924 00:14:14.087984 3435 tablet_bootstrap.cc:654] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.088296 3435 log.cc:824] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:14.088337 3436 tablet_bootstrap.cc:654] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.088584 3434 tablet_bootstrap.cc:492] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: No bootstrap required, opened a new log
I20260924 00:14:14.088675 3434 ts_tablet_manager.cc:1397] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:14.088672 3436 log.cc:824] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:14.089087 3435 tablet_bootstrap.cc:492] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: No bootstrap required, opened a new log
I20260924 00:14:14.089165 3435 ts_tablet_manager.cc:1397] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:14.089423 3436 tablet_bootstrap.cc:492] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: No bootstrap required, opened a new log
I20260924 00:14:14.089493 3436 ts_tablet_manager.cc:1397] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:14.090682 3434 raft_consensus.cc:348] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.090854 3434 raft_consensus.cc:374] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.090888 3434 raft_consensus.cc:729] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacc15bff4d54b79828fd42286619e46, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.091023 3434 consensus_queue.cc:260] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.091087 3435 raft_consensus.cc:348] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.091269 3435 raft_consensus.cc:374] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.091310 3434 ts_tablet_manager.cc:1428] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:14.091311 3435 raft_consensus.cc:729] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.091419 3411 heartbeater.cc:503] Master 127.1.44.190:34719 was elected leader, sending a full tablet report...
I20260924 00:14:14.091419 3435 consensus_queue.cc:260] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.091392 3436 raft_consensus.cc:348] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.091549 3436 raft_consensus.cc:374] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.091593 3436 raft_consensus.cc:729] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.091670 3435 ts_tablet_manager.cc:1428] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:14:14.091729 3436 consensus_queue.cc:260] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.091980 3436 ts_tablet_manager.cc:1428] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:14:14.092052 3131 heartbeater.cc:503] Master 127.1.44.190:34719 was elected leader, sending a full tablet report...
I20260924 00:14:14.092101 3271 heartbeater.cc:503] Master 127.1.44.190:34719 was elected leader, sending a full tablet report...
I20260924 00:14:14.098968 3442 raft_consensus.cc:482] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:14.099048 3442 raft_consensus.cc:504] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.099388 3442 leader_election.cc:288] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:14.102913 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.099604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:44665 (local address 127.1.44.130:39951)
0924 00:14:14.099661 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.099663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.099673 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:14.099783 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.099783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.099793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.099851 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.100013 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.100713 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.102051 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.102572 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.102646 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.102654 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.102819 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.102852 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 00:14:14.102926 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.099736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:59951 (local address 127.1.44.131:41425)
0924 00:14:14.099776 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.099779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.099860 (+ 81us) server_negotiation.cc:407] Connection header received
0924 00:14:14.099890 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.099891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.099899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.099951 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.100176 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.100869 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.102051 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.102573 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.102646 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.102654 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.102819 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.102852 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:14.102931 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.099537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.1.44.129:44665)
0924 00:14:14.099603 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.099615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.099666 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.099853 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.099854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.099856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.099939 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.099940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.100785 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.100786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.101954 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.101957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.102655 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.102656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.102743 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.102745 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.102802 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.102844 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":6}
I20260924 00:14:14.102944 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.099669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41425 (local address 127.1.44.129:59951)
0924 00:14:14.099797 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.099806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.099851 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.100052 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.100053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.100054 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.100110 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.100111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.100930 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.100931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.101977 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.101978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.102655 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.102656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.102738 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.102742 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.102802 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.102844 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":7,"thread_start_us":79,"threads_started":1}
I20260924 00:14:14.103366 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b0796b6d29842e5a16ee7e3b600f570" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" is_pre_election: true
I20260924 00:14:14.103495 3225 raft_consensus.cc:2454] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 0.
I20260924 00:14:14.103703 3016 leader_election.cc:302] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, b1ce649fabd14a6ba0799c40347487d7; no voters:
I20260924 00:14:14.103826 3442 raft_consensus.cc:2781] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:14.103899 3442 raft_consensus.cc:482] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:14.103924 3442 raft_consensus.cc:3037] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.103994 3365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b0796b6d29842e5a16ee7e3b600f570" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46" is_pre_election: true
I20260924 00:14:14.104090 3365 raft_consensus.cc:2454] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 0.
I20260924 00:14:14.104658 3442 raft_consensus.cc:504] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.104789 3442 leader_election.cc:288] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 election: Requested vote from peers 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:14.104964 3365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b0796b6d29842e5a16ee7e3b600f570" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46"
I20260924 00:14:14.104988 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b0796b6d29842e5a16ee7e3b600f570" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6"
I20260924 00:14:14.105036 3365 raft_consensus.cc:3037] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.105069 3225 raft_consensus.cc:3037] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.105669 3365 raft_consensus.cc:2454] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 1.
I20260924 00:14:14.105798 3225 raft_consensus.cc:2454] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 1.
I20260924 00:14:14.105847 3016 leader_election.cc:302] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1ce649fabd14a6ba0799c40347487d7, cacc15bff4d54b79828fd42286619e46; no voters:
I20260924 00:14:14.105931 3442 raft_consensus.cc:2781] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:14.106117 3442 raft_consensus.cc:686] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 1 LEADER]: Becoming Leader. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Running, Role: LEADER
I20260924 00:14:14.106200 3442 consensus_queue.cc:237] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.106902 2928 catalog_manager.cc:5495] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "b1ce649fabd14a6ba0799c40347487d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } health_report { overall_health: UNKNOWN } } }
Sep 24 00:14:14 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208853, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
W20260924 00:14:14.136886 3272 tablet.cc:2369] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:14.138736 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.132024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.1.44.190:34719)
0924 00:14:14.132096 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.132099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.132222 (+ 123us) server_negotiation.cc:407] Connection header received
0924 00:14:14.132303 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.132303 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.132316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.132381 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:14.132849 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.133327 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.134000 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.134180 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.134270 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.136355 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:14.136356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:14.136356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:14.136363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:14.137749 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.138095 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.138096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.138096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.138149 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:14.138320 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:14.138321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:14.138321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:14.138420 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:14.138471 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.138602 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.138667 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 00:14:14.139011 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.131920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34719 (local address 127.0.0.1:40398)
0924 00:14:14.132145 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.132155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.132242 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.132397 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.132398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.132714 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:14.132758 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.132760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.133405 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.133406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.133917 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.133918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.134282 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.134283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.134339 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.134837 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:14.134842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:14.136265 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:14.137825 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:14.137832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:14.137834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:14.138023 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:14.138209 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:14.138210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:14.138210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:14.138266 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:14.138475 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:14.138480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:14.138523 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.138586 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.138735 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":63,"thread_start_us":119,"threads_started":1}
I20260924 00:14:14.140918 2928 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:40398:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 00:14:14.141145 2928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:14.142697 3205 tablet_service.cc:1467] Processing CreateTablet for tablet e0fb89846679415f8aca4b8487cc0260 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3679e081a0a04d7b8aaec1f7a5252f10]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:14.142736 3345 tablet_service.cc:1467] Processing CreateTablet for tablet e0fb89846679415f8aca4b8487cc0260 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3679e081a0a04d7b8aaec1f7a5252f10]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:14.142899 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fb89846679415f8aca4b8487cc0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.142838 3065 tablet_service.cc:1467] Processing CreateTablet for tablet e0fb89846679415f8aca4b8487cc0260 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3679e081a0a04d7b8aaec1f7a5252f10]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:14.142905 3205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fb89846679415f8aca4b8487cc0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.142985 3065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0fb89846679415f8aca4b8487cc0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.144343 3434 tablet_bootstrap.cc:492] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: Bootstrap starting.
I20260924 00:14:14.144476 3435 tablet_bootstrap.cc:492] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: Bootstrap starting.
I20260924 00:14:14.144498 3436 tablet_bootstrap.cc:492] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: Bootstrap starting.
I20260924 00:14:14.144893 3436 tablet_bootstrap.cc:654] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.145009 3434 tablet_bootstrap.cc:654] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.145188 3435 tablet_bootstrap.cc:654] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.145610 3436 tablet_bootstrap.cc:492] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: No bootstrap required, opened a new log
I20260924 00:14:14.145677 3436 ts_tablet_manager.cc:1397] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:14.145812 3436 raft_consensus.cc:348] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.145876 3436 raft_consensus.cc:374] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.145891 3436 raft_consensus.cc:729] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.145951 3434 tablet_bootstrap.cc:492] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: No bootstrap required, opened a new log
I20260924 00:14:14.145946 3436 consensus_queue.cc:260] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.146004 3434 ts_tablet_manager.cc:1397] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 00:14:14.146031 3436 ts_tablet_manager.cc:1428] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.146154 3434 raft_consensus.cc:348] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.146224 3434 raft_consensus.cc:374] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.146255 3434 raft_consensus.cc:729] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacc15bff4d54b79828fd42286619e46, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.146304 3434 consensus_queue.cc:260] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.146409 3434 ts_tablet_manager.cc:1428] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.146432 3435 tablet_bootstrap.cc:492] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: No bootstrap required, opened a new log
I20260924 00:14:14.146471 3435 ts_tablet_manager.cc:1397] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:14.146615 3435 raft_consensus.cc:348] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.146682 3435 raft_consensus.cc:374] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.146704 3435 raft_consensus.cc:729] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.146746 3435 consensus_queue.cc:260] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.146818 3435 ts_tablet_manager.cc:1428] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.146772 3442 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.146853 3442 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:14.146965 3440 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.147075 3440 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:14.147246 3441 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.147330 3441 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 00:14:14.238958 3132 tablet.cc:2369] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:14.244733 3441 raft_consensus.cc:482] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:14.244824 3441 raft_consensus.cc:504] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.245157 3441 leader_election.cc:288] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:14.248013 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.245345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41425 (local address 127.1.44.130:50153)
0924 00:14:14.245414 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.245427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.245478 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.245655 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.245656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.245657 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.245735 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.245737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.246418 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.246419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.247185 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.247186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.247761 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.247762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.247845 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.247848 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.247901 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.247941 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 00:14:14.248042 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.245421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:50153 (local address 127.1.44.131:41425)
0924 00:14:14.245462 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.245465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.245474 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:14.245542 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.245544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.245554 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.245624 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.245819 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.246331 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.247282 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.247683 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.247738 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.247745 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.247921 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.247959 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:14.248175 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.245379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:57225 (local address 127.1.44.129:46323)
0924 00:14:14.245427 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.245429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.245641 (+ 212us) server_negotiation.cc:407] Connection header received
0924 00:14:14.245673 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.245674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.245686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.245739 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.245911 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.246354 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.247378 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.247768 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.247840 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.247848 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.248046 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.248095 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 00:14:14.248185 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.245345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:46323 (local address 127.1.44.130:57225)
0924 00:14:14.245561 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.245571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.245627 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.245750 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.245750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.245751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.245844 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.245846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.246470 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.246471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.247297 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.247299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.247868 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.247869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.247956 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.247960 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.248043 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.248089 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":50,"thread_start_us":101,"threads_started":1}
I20260924 00:14:14.248248 3365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fb89846679415f8aca4b8487cc0260" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46" is_pre_election: true
I20260924 00:14:14.248324 3365 raft_consensus.cc:2454] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 0.
I20260924 00:14:14.248528 3156 leader_election.cc:302] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, cacc15bff4d54b79828fd42286619e46; no voters:
I20260924 00:14:14.248572 3085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fb89846679415f8aca4b8487cc0260" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ce649fabd14a6ba0799c40347487d7" is_pre_election: true
I20260924 00:14:14.248667 3441 raft_consensus.cc:2781] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:14.248678 3085 raft_consensus.cc:2454] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 0.
I20260924 00:14:14.248714 3441 raft_consensus.cc:482] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:14.248731 3441 raft_consensus.cc:3037] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.249460 3441 raft_consensus.cc:504] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.249594 3441 leader_election.cc:288] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 election: Requested vote from peers b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:14.249754 3085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fb89846679415f8aca4b8487cc0260" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ce649fabd14a6ba0799c40347487d7"
I20260924 00:14:14.249842 3085 raft_consensus.cc:3037] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.249825 3365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0fb89846679415f8aca4b8487cc0260" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46"
I20260924 00:14:14.249898 3365 raft_consensus.cc:3037] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.250655 3365 raft_consensus.cc:2454] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 1.
I20260924 00:14:14.250654 3085 raft_consensus.cc:2454] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 1.
I20260924 00:14:14.250908 3156 leader_election.cc:302] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, cacc15bff4d54b79828fd42286619e46; no voters:
I20260924 00:14:14.251026 3441 raft_consensus.cc:2781] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:14.251258 3441 raft_consensus.cc:686] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 LEADER]: Becoming Leader. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Running, Role: LEADER
I20260924 00:14:14.251366 3441 consensus_queue.cc:237] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.251833 3456 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: TxnStatusTablet state changed. Reason: New leader 325c935b654d43c5b7a73a8ab4bf0bf6. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.251915 3456 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:14.252080 3458 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 00:14:14.252190 3458 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 00:14:14.252184 2927 catalog_manager.cc:5495] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130). New cstate: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:14.264621 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.261779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.0.0.1:45938)
0924 00:14:14.261857 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.261870 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.261920 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.262097 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.262097 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.262098 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:14.262141 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.262143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.263053 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.263054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.263675 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.263677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.264164 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.264165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.264214 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.264228 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.264493 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.264497 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.264521 (+ 24us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.264563 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 00:14:14.264644 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.261855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.1.44.130:39951)
0924 00:14:14.261917 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.261919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.261929 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:14.261991 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.261992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.262010 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.262096 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.262182 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.262963 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.263772 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.264090 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.264159 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.264315 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.264433 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.264515 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.264531 (+ 16us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.264551 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:14:14.266278 3085 raft_consensus.cc:1261] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Refusing update from remote peer 325c935b654d43c5b7a73a8ab4bf0bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:14.266319 3365 raft_consensus.cc:1261] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Refusing update from remote peer 325c935b654d43c5b7a73a8ab4bf0bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:14.266570 3456 consensus_queue.cc:1035] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:14.266693 3441 consensus_queue.cc:1035] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:14.267733 3442 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: TxnStatusTablet state changed. Reason: New leader 325c935b654d43c5b7a73a8ab4bf0bf6. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.267825 3442 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:14.268026 3440 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: TxnStatusTablet state changed. Reason: New leader 325c935b654d43c5b7a73a8ab4bf0bf6. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.268112 3440 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:14.268558 3441 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.268652 3441 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:14.269245 3456 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.269320 3456 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P 325c935b654d43c5b7a73a8ab4bf0bf6: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:14.269896 3440 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.269994 3440 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P cacc15bff4d54b79828fd42286619e46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:14.269950 3462 mvcc.cc:205] Tried to move back new op lower bound from 7332695467072401408 to 7332695467014279168. Current Snapshot: MvccSnapshot[applied={T|T < 7332695467072401408}]
I20260924 00:14:14.270756 3442 tablet_replica.cc:439] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } } }
I20260924 00:14:14.270839 3442 tablet_replica.cc:442] T e0fb89846679415f8aca4b8487cc0260 P b1ce649fabd14a6ba0799c40347487d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:14.274221 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.271566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.1.44.129:46323)
0924 00:14:14.271608 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.271610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.271619 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:14.271685 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.271686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.271702 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.271767 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.271899 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.272631 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.273399 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.273704 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.273776 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.273910 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.274030 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.274105 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.274128 (+ 23us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.274152 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:14.279114 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.276797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:55677 (local address 127.1.44.130:39951)
0924 00:14:14.276842 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.276843 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.276922 (+ 79us) server_negotiation.cc:407] Connection header received
0924 00:14:14.276997 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.276998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.277008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.277082 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.277187 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.277678 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.278424 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.278621 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.278692 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.278876 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.278911 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.278970 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.279024 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.279040 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:14:14.279139 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.276730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.1.44.129:55677)
0924 00:14:14.276864 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.276874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.276937 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.277085 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.277086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.277087 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:14.277131 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.277132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.277752 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.277753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.278334 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.278336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.278691 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.278692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.278755 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.278768 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.278971 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.278973 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.279021 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.279060 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260924 00:14:14.281710 3365 raft_consensus.cc:1261] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Refusing update from remote peer b1ce649fabd14a6ba0799c40347487d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:14.281791 3225 raft_consensus.cc:1261] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Refusing update from remote peer b1ce649fabd14a6ba0799c40347487d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:14.281970 3444 consensus_queue.cc:1035] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:14.282094 3442 consensus_queue.cc:1035] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:14.284507 3461 mvcc.cc:205] Tried to move back new op lower bound from 7332695467136299008 to 7332695466419507200. Current Snapshot: MvccSnapshot[applied={T|T < 7332695467136299008}]
I20260924 00:14:14.290851 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.288054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:46323 (local address 127.1.44.130:34971)
0924 00:14:14.288110 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.288122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.288170 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.288308 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.288309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.288310 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:14.288360 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.288361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.289325 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.289326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.290006 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.290008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.290471 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.290472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.290549 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.290562 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.290739 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.290740 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.290774 (+ 34us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.290803 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 00:14:14.290885 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.288126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:34971 (local address 127.1.44.129:46323)
0924 00:14:14.288162 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.288164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.288172 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:14.288232 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.288233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.288244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.288296 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.288423 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.289197 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.290078 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.290394 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.290469 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.290646 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.290679 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.290731 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.290777 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.290795 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:14.300995 2928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:14.301102 2928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:14.301807 2928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Sending DeleteTablet for 3 replicas of tablet 6b0796b6d29842e5a16ee7e3b600f570
I20260924 00:14:14.302245 3205 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b0796b6d29842e5a16ee7e3b600f570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:14 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260924 00:14:14.302431 3065 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b0796b6d29842e5a16ee7e3b600f570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:14 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:38694
I20260924 00:14:14.302484 3481 tablet_replica.cc:330] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: stopping tablet replica
I20260924 00:14:14.302593 3481 raft_consensus.cc:2229] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:14.302650 3481 raft_consensus.cc:2258] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:14.302812 3345 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b0796b6d29842e5a16ee7e3b600f570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:14 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:60048
I20260924 00:14:14.303107 3481 ts_tablet_manager.cc:1905] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:14.303484 2928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:14.303622 2928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:14.304708 3482 tablet_replica.cc:330] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: stopping tablet replica
I20260924 00:14:14.304952 3481 ts_tablet_manager.cc:1918] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:14.305023 3481 log.cc:1196] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/6b0796b6d29842e5a16ee7e3b600f570
I20260924 00:14:14.305230 3205 tablet_service.cc:1467] Processing CreateTablet for tablet 0bdd360a51e34acda8b85e865fad9740 (DEFAULT_TABLE table=test-table [id=d6e628963ac0440b80edc4359ea230b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:14.305281 3481 ts_tablet_manager.cc:1939] T 6b0796b6d29842e5a16ee7e3b600f570 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting consensus metadata
I20260924 00:14:14.305332 3205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bdd360a51e34acda8b85e865fad9740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.306489 3345 tablet_service.cc:1467] Processing CreateTablet for tablet 0bdd360a51e34acda8b85e865fad9740 (DEFAULT_TABLE table=test-table [id=d6e628963ac0440b80edc4359ea230b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:14.306596 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bdd360a51e34acda8b85e865fad9740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.307294 3435 tablet_bootstrap.cc:492] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Bootstrap starting.
I20260924 00:14:14.307685 3435 tablet_bootstrap.cc:654] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.308295 3435 tablet_bootstrap.cc:492] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: No bootstrap required, opened a new log
I20260924 00:14:14.308341 3435 ts_tablet_manager.cc:1397] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:14.308466 2912 catalog_manager.cc:4841] TS 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951): tablet 6b0796b6d29842e5a16ee7e3b600f570 (table test-table [id=2097a306df76465d9127617a190fdb30]) successfully deleted
I20260924 00:14:14.308480 3435 raft_consensus.cc:348] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.308544 3435 raft_consensus.cc:374] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.308565 3435 raft_consensus.cc:729] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.308605 3435 consensus_queue.cc:260] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.308698 3435 ts_tablet_manager.cc:1428] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.308933 3483 tablet_replica.cc:330] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: stopping tablet replica
I20260924 00:14:14.309059 3483 raft_consensus.cc:2229] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:14.309118 3483 raft_consensus.cc:2258] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:14.309293 3482 raft_consensus.cc:2229] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:14.309397 3482 raft_consensus.cc:2258] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:14.309603 3434 tablet_bootstrap.cc:492] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Bootstrap starting.
I20260924 00:14:14.309952 3483 ts_tablet_manager.cc:1905] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:14.309990 3434 tablet_bootstrap.cc:654] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.305148 3065 tablet_service.cc:1467] Processing CreateTablet for tablet 0bdd360a51e34acda8b85e865fad9740 (DEFAULT_TABLE table=test-table [id=d6e628963ac0440b80edc4359ea230b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:14.310707 3065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bdd360a51e34acda8b85e865fad9740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:14.311553 3483 ts_tablet_manager.cc:1918] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:14.311612 3483 log.cc:1196] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/6b0796b6d29842e5a16ee7e3b600f570
I20260924 00:14:14.311844 3483 ts_tablet_manager.cc:1939] T 6b0796b6d29842e5a16ee7e3b600f570 P cacc15bff4d54b79828fd42286619e46: Deleting consensus metadata
I20260924 00:14:14.311964 3436 tablet_bootstrap.cc:492] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Bootstrap starting.
I20260924 00:14:14.312351 3436 tablet_bootstrap.cc:654] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:14.312597 3482 ts_tablet_manager.cc:1905] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:14.313593 3436 tablet_bootstrap.cc:492] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: No bootstrap required, opened a new log
I20260924 00:14:14.313648 3436 ts_tablet_manager.cc:1397] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:14.313788 3436 raft_consensus.cc:348] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.313859 3436 raft_consensus.cc:374] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.313887 3436 raft_consensus.cc:729] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.313934 3436 consensus_queue.cc:260] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.314060 3436 ts_tablet_manager.cc:1428] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:14.314182 3482 ts_tablet_manager.cc:1918] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:14.314302 3482 log.cc:1196] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/6b0796b6d29842e5a16ee7e3b600f570
I20260924 00:14:14.314543 3482 ts_tablet_manager.cc:1939] T 6b0796b6d29842e5a16ee7e3b600f570 P b1ce649fabd14a6ba0799c40347487d7: Deleting consensus metadata
I20260924 00:14:14.314913 2915 catalog_manager.cc:4841] TS b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323): tablet 6b0796b6d29842e5a16ee7e3b600f570 (table test-table [id=2097a306df76465d9127617a190fdb30]) successfully deleted
I20260924 00:14:14.315245 3434 tablet_bootstrap.cc:492] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: No bootstrap required, opened a new log
I20260924 00:14:14.315305 3434 ts_tablet_manager.cc:1397] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260924 00:14:14.315433 3434 raft_consensus.cc:348] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.315490 3434 raft_consensus.cc:374] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:14.315508 3434 raft_consensus.cc:729] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacc15bff4d54b79828fd42286619e46, State: Initialized, Role: FOLLOWER
I20260924 00:14:14.315544 2912 catalog_manager.cc:4841] TS cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425): tablet 6b0796b6d29842e5a16ee7e3b600f570 (table test-table [id=2097a306df76465d9127617a190fdb30]) successfully deleted
I20260924 00:14:14.315546 3434 consensus_queue.cc:260] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.315644 3434 ts_tablet_manager.cc:1428] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260924 00:14:14.541234 3412 tablet.cc:2369] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:14.550313 3440 raft_consensus.cc:482] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:14.550395 3440 raft_consensus.cc:504] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.550671 3440 leader_election.cc:288] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323), 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951)
I20260924 00:14:14.553531 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.550931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:54629 (local address 127.1.44.130:39951)
0924 00:14:14.550985 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.550988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.551035 (+ 47us) server_negotiation.cc:407] Connection header received
0924 00:14:14.551133 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.551134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.551144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.551238 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.551387 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.551860 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.552808 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.553159 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.553261 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.553271 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.553432 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.553460 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:14:14.553546 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.550922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44451 (local address 127.1.44.129:46323)
0924 00:14:14.550992 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.550994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.551102 (+ 108us) server_negotiation.cc:407] Connection header received
0924 00:14:14.551167 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.551168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.551178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.551265 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:14.551451 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.551902 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.552814 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.553169 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.553281 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.553288 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.553448 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.553478 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 00:14:14.553577 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.550853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.1.44.131:54629)
0924 00:14:14.550978 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.550990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.551051 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.551242 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.551243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.551245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.551317 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.551318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.551941 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.551942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.552714 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.552716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.553260 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.553261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.553336 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.553339 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.553425 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.553461 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":52,"threads_started":1}
I20260924 00:14:14.553579 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.550853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:46323 (local address 127.1.44.131:44451)
0924 00:14:14.550993 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.551005 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.551100 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.551312 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.551312 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.551314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:14.551367 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.551368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.551981 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.551983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.552717 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.552719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.553285 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.553286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.553360 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.553363 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.553442 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.553479 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":14,"thread_start_us":42,"threads_started":1}
I20260924 00:14:14.553841 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bdd360a51e34acda8b85e865fad9740" candidate_uuid: "cacc15bff4d54b79828fd42286619e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" is_pre_election: true
I20260924 00:14:14.553936 3225 raft_consensus.cc:2454] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cacc15bff4d54b79828fd42286619e46 in term 0.
I20260924 00:14:14.553928 3085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bdd360a51e34acda8b85e865fad9740" candidate_uuid: "cacc15bff4d54b79828fd42286619e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ce649fabd14a6ba0799c40347487d7" is_pre_election: true
I20260924 00:14:14.554021 3085 raft_consensus.cc:2454] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cacc15bff4d54b79828fd42286619e46 in term 0.
I20260924 00:14:14.554132 3296 leader_election.cc:302] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, cacc15bff4d54b79828fd42286619e46; no voters:
I20260924 00:14:14.554268 3440 raft_consensus.cc:2781] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:14.554314 3440 raft_consensus.cc:482] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:14.554344 3440 raft_consensus.cc:3037] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.555013 3440 raft_consensus.cc:504] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.555157 3440 leader_election.cc:288] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [CANDIDATE]: Term 1 election: Requested vote from peers b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323), 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951)
I20260924 00:14:14.555334 3085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bdd360a51e34acda8b85e865fad9740" candidate_uuid: "cacc15bff4d54b79828fd42286619e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ce649fabd14a6ba0799c40347487d7"
I20260924 00:14:14.555343 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bdd360a51e34acda8b85e865fad9740" candidate_uuid: "cacc15bff4d54b79828fd42286619e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6"
I20260924 00:14:14.555414 3225 raft_consensus.cc:3037] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.555414 3085 raft_consensus.cc:3037] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:14.556174 3085 raft_consensus.cc:2454] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cacc15bff4d54b79828fd42286619e46 in term 1.
I20260924 00:14:14.556174 3225 raft_consensus.cc:2454] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cacc15bff4d54b79828fd42286619e46 in term 1.
I20260924 00:14:14.556334 3296 leader_election.cc:302] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, cacc15bff4d54b79828fd42286619e46; no voters:
I20260924 00:14:14.556442 3440 raft_consensus.cc:2781] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:14.556584 3440 raft_consensus.cc:686] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 1 LEADER]: Becoming Leader. State: Replica: cacc15bff4d54b79828fd42286619e46, State: Running, Role: LEADER
I20260924 00:14:14.556658 3440 consensus_queue.cc:237] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:14.557344 2928 catalog_manager.cc:5495] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 reported cstate change: term changed from 0 to 1, leader changed from <none> to cacc15bff4d54b79828fd42286619e46 (127.1.44.131). New cstate: current_term: 1 leader_uuid: "cacc15bff4d54b79828fd42286619e46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } health_report { overall_health: HEALTHY } } }
I20260924 00:14:14.598030 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.595635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.1.44.131:41425)
0924 00:14:14.595681 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.595683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.595692 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:14.595776 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.595777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.595798 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.595892 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.595994 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.596476 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.597208 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.597413 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.597489 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.597655 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.597776 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.597861 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.597901 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.597918 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:14.602679 3431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.600657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:36199 (local address 127.1.44.130:39951)
0924 00:14:14.600695 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.600697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.600802 (+ 105us) server_negotiation.cc:407] Connection header received
0924 00:14:14.600842 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.600842 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.600851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.600890 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.600995 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.601420 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.602120 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.602305 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.602366 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.602496 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.602521 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.602559 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.602610 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.602625 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:14.602707 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.600586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.1.44.131:36199)
0924 00:14:14.600731 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.600741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.600798 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.600905 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.600906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.600907 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:14.600952 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.600953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.601473 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.601473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.602034 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.602036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.602375 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.602376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.602420 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.602431 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.602571 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.602572 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.602606 (+ 34us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.602636 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260924 00:14:14.604566 3085 raft_consensus.cc:1261] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Refusing update from remote peer cacc15bff4d54b79828fd42286619e46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:14.604597 3225 raft_consensus.cc:1261] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Refusing update from remote peer cacc15bff4d54b79828fd42286619e46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:14.604820 3489 consensus_queue.cc:1035] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:14.604940 3440 consensus_queue.cc:1035] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:14.611536 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.609320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:43871 (local address 127.1.44.131:41425)
0924 00:14:14.609366 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:14.609368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:14.609392 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:14:14.609447 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:14.609448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:14.609454 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:14.609520 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:14.609635 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.610182 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.610913 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.611090 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.611140 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.611318 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.611355 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.611407 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:14.611445 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 00:14:14.611467 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 00:14:14.611583 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:14.609307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41425 (local address 127.1.44.130:43871)
0924 00:14:14.609350 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:14.609360 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:14.609406 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:14.609523 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:14.609524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:14.609525 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:14.609562 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.609563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.610250 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.610251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.610846 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:14.610849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:14.611169 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:14.611170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:14.611247 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:14.611252 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:14.611401 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:14.611403 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:14.611442 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 00:14:14.611487 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
W20260924 00:14:14.611938 3464 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 00:14:16.130249 2926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:16.130373 2926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:16.131134 2926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Sending DeleteTablet for 3 replicas of tablet 0bdd360a51e34acda8b85e865fad9740
I20260924 00:14:16.131421 3205 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bdd360a51e34acda8b85e865fad9740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:16 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260924 00:14:16.131429 3065 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bdd360a51e34acda8b85e865fad9740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:16 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:38694
I20260924 00:14:16.131457 3345 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bdd360a51e34acda8b85e865fad9740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:16 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:60048
I20260924 00:14:16.131628 3520 tablet_replica.cc:330] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: stopping tablet replica
I20260924 00:14:16.131687 3520 raft_consensus.cc:2229] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:16.131771 3520 raft_consensus.cc:2258] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:16.131867 3521 tablet_replica.cc:330] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: stopping tablet replica
I20260924 00:14:16.131918 3521 raft_consensus.cc:2229] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:16.131954 3521 raft_consensus.cc:2258] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:16.132040 3520 ts_tablet_manager.cc:1905] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:16.132189 3521 ts_tablet_manager.cc:1905] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:16.133539 3520 ts_tablet_manager.cc:1918] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:14:16.133599 3520 log.cc:1196] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/0bdd360a51e34acda8b85e865fad9740
I20260924 00:14:16.133846 3520 ts_tablet_manager.cc:1939] T 0bdd360a51e34acda8b85e865fad9740 P cacc15bff4d54b79828fd42286619e46: Deleting consensus metadata
I20260924 00:14:16.134007 3521 ts_tablet_manager.cc:1918] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:14:16.134064 3521 log.cc:1196] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/0bdd360a51e34acda8b85e865fad9740
I20260924 00:14:16.134223 2912 catalog_manager.cc:4841] TS cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425): tablet 0bdd360a51e34acda8b85e865fad9740 (table test-table [id=d6e628963ac0440b80edc4359ea230b4]) successfully deleted
I20260924 00:14:16.134276 3521 ts_tablet_manager.cc:1939] T 0bdd360a51e34acda8b85e865fad9740 P b1ce649fabd14a6ba0799c40347487d7: Deleting consensus metadata
I20260924 00:14:16.134689 2915 catalog_manager.cc:4841] TS b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323): tablet 0bdd360a51e34acda8b85e865fad9740 (table test-table [id=d6e628963ac0440b80edc4359ea230b4]) successfully deleted
I20260924 00:14:16.136540 3519 tablet_replica.cc:330] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: stopping tablet replica
I20260924 00:14:16.136620 3519 raft_consensus.cc:2229] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:16.136665 3519 raft_consensus.cc:2258] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:16.136871 3519 ts_tablet_manager.cc:1905] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:16.138403 3519 ts_tablet_manager.cc:1918] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:14:16.138480 3519 log.cc:1196] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/0bdd360a51e34acda8b85e865fad9740
I20260924 00:14:16.138695 3519 ts_tablet_manager.cc:1939] T 0bdd360a51e34acda8b85e865fad9740 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting consensus metadata
I20260924 00:14:16.139103 2912 catalog_manager.cc:4841] TS 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951): tablet 0bdd360a51e34acda8b85e865fad9740 (table test-table [id=d6e628963ac0440b80edc4359ea230b4]) successfully deleted
Sep 24 00:14:16 dist-test-slave-cm7b krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208856, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:14:16 dist-test-slave-cm7b krb5kdc[2883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208856, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:16.154984 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:16.148382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.1.44.190:34719)
0924 00:14:16.148561 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:16.148563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:16.148572 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:16.148595 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:16.148596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:16.148606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:16.148661 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:16.149170 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.149663 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.150343 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.150523 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.150572 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:16.152629 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:16.152630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:16.152630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:16.152636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:16.153917 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:16.154259 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:16.154259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:16.154260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:16.154308 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:16.154499 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:16.154500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:16.154500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:16.154611 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:16.154699 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:16.154847 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 00:14:16.154910 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260924 00:14:16.156617 2926 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:16.156787 2926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:16.158172 3205 tablet_service.cc:1467] Processing CreateTablet for tablet 7616fe510b11440c861bc597e298c0d5 (DEFAULT_TABLE table=test-table [id=3717b087e6124ceeb40cca5d8f886e66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:16.158172 3065 tablet_service.cc:1467] Processing CreateTablet for tablet 7616fe510b11440c861bc597e298c0d5 (DEFAULT_TABLE table=test-table [id=3717b087e6124ceeb40cca5d8f886e66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:16.158198 3345 tablet_service.cc:1467] Processing CreateTablet for tablet 7616fe510b11440c861bc597e298c0d5 (DEFAULT_TABLE table=test-table [id=3717b087e6124ceeb40cca5d8f886e66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:16.158296 3065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7616fe510b11440c861bc597e298c0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:16.158295 3205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7616fe510b11440c861bc597e298c0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:16.158334 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7616fe510b11440c861bc597e298c0d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:16.159593 3531 tablet_bootstrap.cc:492] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Bootstrap starting.
I20260924 00:14:16.159659 3532 tablet_bootstrap.cc:492] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Bootstrap starting.
I20260924 00:14:16.159796 3530 tablet_bootstrap.cc:492] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Bootstrap starting.
I20260924 00:14:16.160028 3532 tablet_bootstrap.cc:654] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:16.160037 3531 tablet_bootstrap.cc:654] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:16.160156 3530 tablet_bootstrap.cc:654] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:16.160594 3531 tablet_bootstrap.cc:492] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: No bootstrap required, opened a new log
I20260924 00:14:16.160648 3531 ts_tablet_manager.cc:1397] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:16.160749 3532 tablet_bootstrap.cc:492] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: No bootstrap required, opened a new log
I20260924 00:14:16.160769 3531 raft_consensus.cc:348] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.160830 3532 ts_tablet_manager.cc:1397] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:16.160846 3531 raft_consensus.cc:374] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:16.160866 3531 raft_consensus.cc:729] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Initialized, Role: FOLLOWER
I20260924 00:14:16.160915 3531 consensus_queue.cc:260] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.160943 3532 raft_consensus.cc:348] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.161015 3532 raft_consensus.cc:374] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:16.161046 3532 raft_consensus.cc:729] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Initialized, Role: FOLLOWER
I20260924 00:14:16.161098 3532 consensus_queue.cc:260] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.161120 3530 tablet_bootstrap.cc:492] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: No bootstrap required, opened a new log
I20260924 00:14:16.161158 3530 ts_tablet_manager.cc:1397] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:16.161265 3532 ts_tablet_manager.cc:1428] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:16.161254 3530 raft_consensus.cc:348] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.161307 3530 raft_consensus.cc:374] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:16.161325 3530 raft_consensus.cc:729] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacc15bff4d54b79828fd42286619e46, State: Initialized, Role: FOLLOWER
I20260924 00:14:16.161365 3530 consensus_queue.cc:260] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.161449 3530 ts_tablet_manager.cc:1428] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:16.161576 3531 ts_tablet_manager.cc:1428] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:16.361763 3500 raft_consensus.cc:482] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:16.361852 3500 raft_consensus.cc:504] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.362026 3500 leader_election.cc:288] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:16.362250 3084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7616fe510b11440c861bc597e298c0d5" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ce649fabd14a6ba0799c40347487d7" is_pre_election: true
I20260924 00:14:16.362259 3364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7616fe510b11440c861bc597e298c0d5" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46" is_pre_election: true
I20260924 00:14:16.362350 3084 raft_consensus.cc:2454] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 0.
I20260924 00:14:16.362351 3364 raft_consensus.cc:2454] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 0.
I20260924 00:14:16.362531 3156 leader_election.cc:302] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, cacc15bff4d54b79828fd42286619e46; no voters:
I20260924 00:14:16.362650 3500 raft_consensus.cc:2781] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:16.362705 3500 raft_consensus.cc:482] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:16.362731 3500 raft_consensus.cc:3037] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:16.363471 3500 raft_consensus.cc:504] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.363610 3500 leader_election.cc:288] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 election: Requested vote from peers b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:16.363735 3084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7616fe510b11440c861bc597e298c0d5" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ce649fabd14a6ba0799c40347487d7"
I20260924 00:14:16.363807 3084 raft_consensus.cc:3037] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:16.363754 3364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7616fe510b11440c861bc597e298c0d5" candidate_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46"
I20260924 00:14:16.363842 3364 raft_consensus.cc:3037] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:16.364431 3084 raft_consensus.cc:2454] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 1.
I20260924 00:14:16.364504 3364 raft_consensus.cc:2454] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 325c935b654d43c5b7a73a8ab4bf0bf6 in term 1.
I20260924 00:14:16.364624 3159 leader_election.cc:302] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, b1ce649fabd14a6ba0799c40347487d7; no voters:
I20260924 00:14:16.364701 3500 raft_consensus.cc:2781] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:16.364753 3500 raft_consensus.cc:686] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 LEADER]: Becoming Leader. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Running, Role: LEADER
I20260924 00:14:16.364814 3500 consensus_queue.cc:237] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.365407 2927 catalog_manager.cc:5495] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130). New cstate: current_term: 1 leader_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:16.388857 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:16.386601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.1.44.130:39951)
0924 00:14:16.386742 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:16.386743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:16.386751 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:16.386769 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:16.386770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:16.386779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:16.386835 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:16.386940 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.387468 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.388173 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.388357 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.388405 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:16.388521 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:16.388553 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:16.388634 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:16.388732 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 00:14:16.388750 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":92,"threads_started":1}
I20260924 00:14:16.392817 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:16.390929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:45135 (local address 127.1.44.130:39951)
0924 00:14:16.390960 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:16.390961 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:16.391005 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:14:16.391057 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:16.391057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:16.391064 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:16.391101 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:16.391181 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.391647 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.392345 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.392508 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.392556 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:16.392655 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:16.392674 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:16.392706 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:16.392747 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 00:14:16.392760 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:16.392887 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:16.390784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39951 (local address 127.1.44.130:45135)
0924 00:14:16.390944 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:16.390954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:16.391019 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:16.391111 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:16.391111 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:16.391113 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:16.391144 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:16.391145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.391709 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.391710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:16.392281 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:16.392283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.392570 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.392570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:16.392606 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:16.392609 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:16.392716 (+ 107us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:16.392716 (+ 0us) client_negotiation.cc:769] Sending connection context
0924 00:14:16.392745 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 00:14:16.392774 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260924 00:14:16.394483 3084 raft_consensus.cc:1261] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Refusing update from remote peer 325c935b654d43c5b7a73a8ab4bf0bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:16.394603 3364 raft_consensus.cc:1261] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Refusing update from remote peer 325c935b654d43c5b7a73a8ab4bf0bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:16.394704 3505 consensus_queue.cc:1035] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:16.394819 3505 consensus_queue.cc:1035] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:16.396476 3501 mvcc.cc:205] Tried to move back new op lower bound from 7332695475790475264 to 7332695475670503424. Current Snapshot: MvccSnapshot[applied={T|T < 7332695475790475264}]
I20260924 00:14:16.404386 2927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:16.404467 2927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:16.405066 2927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Sending DeleteTablet for 3 replicas of tablet 7616fe510b11440c861bc597e298c0d5
I20260924 00:14:16.405326 3065 tablet_service.cc:1514] Processing DeleteTablet for tablet 7616fe510b11440c861bc597e298c0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:16 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:38694
I20260924 00:14:16.405362 3345 tablet_service.cc:1514] Processing DeleteTablet for tablet 7616fe510b11440c861bc597e298c0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:16 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:60048
I20260924 00:14:16.405419 3521 tablet_replica.cc:330] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: stopping tablet replica
I20260924 00:14:16.405431 3520 tablet_replica.cc:330] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: stopping tablet replica
I20260924 00:14:16.405471 3521 raft_consensus.cc:2229] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:16.405480 3520 raft_consensus.cc:2229] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:16.405514 3520 raft_consensus.cc:2258] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:16.405515 3521 raft_consensus.cc:2258] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:16.405745 3205 tablet_service.cc:1514] Processing DeleteTablet for tablet 7616fe510b11440c861bc597e298c0d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:16 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260924 00:14:16.405814 3520 ts_tablet_manager.cc:1905] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:16.405831 3519 tablet_replica.cc:330] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: stopping tablet replica
I20260924 00:14:16.405876 3519 raft_consensus.cc:2229] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:16.405957 3519 raft_consensus.cc:2258] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:16.406181 3521 ts_tablet_manager.cc:1905] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:16.407434 3520 ts_tablet_manager.cc:1918] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:16.407497 3520 log.cc:1196] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/7616fe510b11440c861bc597e298c0d5
I20260924 00:14:16.407706 3520 ts_tablet_manager.cc:1939] T 7616fe510b11440c861bc597e298c0d5 P cacc15bff4d54b79828fd42286619e46: Deleting consensus metadata
I20260924 00:14:16.407719 3521 ts_tablet_manager.cc:1918] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:16.407768 3521 log.cc:1196] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/7616fe510b11440c861bc597e298c0d5
I20260924 00:14:16.407969 3521 ts_tablet_manager.cc:1939] T 7616fe510b11440c861bc597e298c0d5 P b1ce649fabd14a6ba0799c40347487d7: Deleting consensus metadata
I20260924 00:14:16.408013 3519 ts_tablet_manager.cc:1905] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:16.408116 2927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:16.408257 2927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:16.408326 2915 catalog_manager.cc:4841] TS b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323): tablet 7616fe510b11440c861bc597e298c0d5 (table test-table [id=3717b087e6124ceeb40cca5d8f886e66]) successfully deleted
I20260924 00:14:16.409353 3065 tablet_service.cc:1467] Processing CreateTablet for tablet 1e6dce949bc44daebb1b886f041ae8d4 (DEFAULT_TABLE table=test-table [id=c0535d582cb44dbc9c8bb631c9fc1823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:16.409359 3205 tablet_service.cc:1467] Processing CreateTablet for tablet 1e6dce949bc44daebb1b886f041ae8d4 (DEFAULT_TABLE table=test-table [id=c0535d582cb44dbc9c8bb631c9fc1823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:16.409456 3065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e6dce949bc44daebb1b886f041ae8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:16.409495 3205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e6dce949bc44daebb1b886f041ae8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:16.409605 3345 tablet_service.cc:1467] Processing CreateTablet for tablet 1e6dce949bc44daebb1b886f041ae8d4 (DEFAULT_TABLE table=test-table [id=c0535d582cb44dbc9c8bb631c9fc1823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:16.409686 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e6dce949bc44daebb1b886f041ae8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:16.410588 3532 tablet_bootstrap.cc:492] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7: Bootstrap starting.
I20260924 00:14:16.410902 3532 tablet_bootstrap.cc:654] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:16.409495 3519 ts_tablet_manager.cc:1918] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:16.411019 3519 log.cc:1196] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/7616fe510b11440c861bc597e298c0d5
I20260924 00:14:16.411275 3519 ts_tablet_manager.cc:1939] T 7616fe510b11440c861bc597e298c0d5 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting consensus metadata
I20260924 00:14:16.411574 2912 catalog_manager.cc:4841] TS 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951): tablet 7616fe510b11440c861bc597e298c0d5 (table test-table [id=3717b087e6124ceeb40cca5d8f886e66]) successfully deleted
I20260924 00:14:16.411607 3532 tablet_bootstrap.cc:492] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7: No bootstrap required, opened a new log
I20260924 00:14:16.411643 2912 catalog_manager.cc:4841] TS cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425): tablet 7616fe510b11440c861bc597e298c0d5 (table test-table [id=3717b087e6124ceeb40cca5d8f886e66]) successfully deleted
I20260924 00:14:16.411648 3532 ts_tablet_manager.cc:1397] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:16.411674 3530 tablet_bootstrap.cc:492] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Bootstrap starting.
I20260924 00:14:16.411741 3532 raft_consensus.cc:348] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.411798 3532 raft_consensus.cc:374] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:16.411818 3532 raft_consensus.cc:729] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Initialized, Role: FOLLOWER
I20260924 00:14:16.411862 3532 consensus_queue.cc:260] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.411944 3532 ts_tablet_manager.cc:1428] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:16.411906 3531 tablet_bootstrap.cc:492] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Bootstrap starting.
I20260924 00:14:16.412220 3531 tablet_bootstrap.cc:654] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:16.412577 3530 tablet_bootstrap.cc:654] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:16.413116 3531 tablet_bootstrap.cc:492] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: No bootstrap required, opened a new log
I20260924 00:14:16.413141 3530 tablet_bootstrap.cc:492] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: No bootstrap required, opened a new log
I20260924 00:14:16.413184 3530 ts_tablet_manager.cc:1397] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:16.413182 3531 ts_tablet_manager.cc:1397] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:16.413285 3530 raft_consensus.cc:348] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.413291 3531 raft_consensus.cc:348] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.413352 3530 raft_consensus.cc:374] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:16.413371 3530 raft_consensus.cc:729] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacc15bff4d54b79828fd42286619e46, State: Initialized, Role: FOLLOWER
I20260924 00:14:16.413368 3531 raft_consensus.cc:374] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:16.413394 3531 raft_consensus.cc:729] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325c935b654d43c5b7a73a8ab4bf0bf6, State: Initialized, Role: FOLLOWER
I20260924 00:14:16.413410 3530 consensus_queue.cc:260] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.413439 3531 consensus_queue.cc:260] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.413483 3530 ts_tablet_manager.cc:1428] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:16.413529 3531 ts_tablet_manager.cc:1428] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:16.556603 3536 raft_consensus.cc:482] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:16.556690 3536 raft_consensus.cc:504] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.556871 3536 leader_election.cc:288] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:16.557120 3364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6dce949bc44daebb1b886f041ae8d4" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46" is_pre_election: true
I20260924 00:14:16.557116 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6dce949bc44daebb1b886f041ae8d4" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" is_pre_election: true
I20260924 00:14:16.557205 3364 raft_consensus.cc:2454] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 0.
I20260924 00:14:16.557229 3225 raft_consensus.cc:2454] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 0.
I20260924 00:14:16.557372 3016 leader_election.cc:302] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, b1ce649fabd14a6ba0799c40347487d7; no voters:
I20260924 00:14:16.557475 3536 raft_consensus.cc:2781] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:16.557516 3536 raft_consensus.cc:482] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:16.557538 3536 raft_consensus.cc:3037] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:16.558200 3536 raft_consensus.cc:504] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.558332 3536 leader_election.cc:288] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 election: Requested vote from peers 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951), cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425)
I20260924 00:14:16.558495 3225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6dce949bc44daebb1b886f041ae8d4" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6"
I20260924 00:14:16.558488 3364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6dce949bc44daebb1b886f041ae8d4" candidate_uuid: "b1ce649fabd14a6ba0799c40347487d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cacc15bff4d54b79828fd42286619e46"
I20260924 00:14:16.558564 3364 raft_consensus.cc:3037] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:16.558564 3225 raft_consensus.cc:3037] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:16.559329 3364 raft_consensus.cc:2454] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 1.
I20260924 00:14:16.559329 3225 raft_consensus.cc:2454] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1ce649fabd14a6ba0799c40347487d7 in term 1.
I20260924 00:14:16.559514 3016 leader_election.cc:302] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 325c935b654d43c5b7a73a8ab4bf0bf6, b1ce649fabd14a6ba0799c40347487d7; no voters:
I20260924 00:14:16.559621 3536 raft_consensus.cc:2781] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:16.559731 3536 raft_consensus.cc:686] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 1 LEADER]: Becoming Leader. State: Replica: b1ce649fabd14a6ba0799c40347487d7, State: Running, Role: LEADER
I20260924 00:14:16.559773 3536 consensus_queue.cc:237] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } }
I20260924 00:14:16.560365 2928 catalog_manager.cc:5495] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "b1ce649fabd14a6ba0799c40347487d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1ce649fabd14a6ba0799c40347487d7" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 46323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:16.597103 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:16.594727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.1.44.129:46323)
0924 00:14:16.594909 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:16.594911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:16.594920 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:16.594943 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:16.594944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:16.594952 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:16.595026 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:16.595135 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.595641 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.596387 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:16.596606 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:16.596682 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:16.596836 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:16.596877 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:16.596938 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:16.596999 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 00:14:16.597018 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260924 00:14:16.599097 3364 raft_consensus.cc:1261] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Refusing update from remote peer b1ce649fabd14a6ba0799c40347487d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:16.599097 3225 raft_consensus.cc:1261] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Refusing update from remote peer b1ce649fabd14a6ba0799c40347487d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:16.599396 3536 consensus_queue.cc:1035] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cacc15bff4d54b79828fd42286619e46" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:16.599495 3549 consensus_queue.cc:1035] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "325c935b654d43c5b7a73a8ab4bf0bf6" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 00:14:16.603394 3502 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 00:14:18.122435 2928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:18.122526 2928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:18.123618 2928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d0cea0f37ba45078ecdbe73ef586394: Sending DeleteTablet for 3 replicas of tablet 1e6dce949bc44daebb1b886f041ae8d4
I20260924 00:14:18.123881 3205 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e6dce949bc44daebb1b886f041ae8d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:18 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260924 00:14:18.123909 3065 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e6dce949bc44daebb1b886f041ae8d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:18 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:38694
I20260924 00:14:18.123953 3345 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e6dce949bc44daebb1b886f041ae8d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:18 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:60048
I20260924 00:14:18.124192 3581 tablet_replica.cc:330] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: stopping tablet replica
I20260924 00:14:18.124193 3580 tablet_replica.cc:330] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7: stopping tablet replica
I20260924 00:14:18.124257 3581 raft_consensus.cc:2229] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:18.124260 3580 raft_consensus.cc:2229] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:18.124286 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3000
I20260924 00:14:18.124322 3580 raft_consensus.cc:2258] T 1e6dce949bc44daebb1b886f041ae8d4 P b1ce649fabd14a6ba0799c40347487d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:18.124330 3581 raft_consensus.cc:2258] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:18.125223 3581 ts_tablet_manager.cc:1905] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:18.126601 3581 ts_tablet_manager.cc:1918] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:14:18.126648 3581 log.cc:1196] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/1e6dce949bc44daebb1b886f041ae8d4
I20260924 00:14:18.126817 3581 ts_tablet_manager.cc:1939] T 1e6dce949bc44daebb1b886f041ae8d4 P cacc15bff4d54b79828fd42286619e46: Deleting consensus metadata
I20260924 00:14:18.127063 2912 catalog_manager.cc:4841] TS cacc15bff4d54b79828fd42286619e46 (127.1.44.131:41425): tablet 1e6dce949bc44daebb1b886f041ae8d4 (table test-table [id=c0535d582cb44dbc9c8bb631c9fc1823]) successfully deleted
I20260924 00:14:18.128490 3579 tablet_replica.cc:330] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: stopping tablet replica
I20260924 00:14:18.128574 3579 raft_consensus.cc:2229] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:18.128619 3579 raft_consensus.cc:2258] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:18.128806 3579 ts_tablet_manager.cc:1905] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:18.130101 3579 ts_tablet_manager.cc:1918] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 00:14:18.130177 3579 log.cc:1196] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/1e6dce949bc44daebb1b886f041ae8d4
I20260924 00:14:18.130399 3579 ts_tablet_manager.cc:1939] T 1e6dce949bc44daebb1b886f041ae8d4 P 325c935b654d43c5b7a73a8ab4bf0bf6: Deleting consensus metadata
I20260924 00:14:18.130688 2912 catalog_manager.cc:4841] TS 325c935b654d43c5b7a73a8ab4bf0bf6 (127.1.44.130:39951): tablet 1e6dce949bc44daebb1b886f041ae8d4 (table test-table [id=c0535d582cb44dbc9c8bb631c9fc1823]) successfully deleted
W20260924 00:14:18.131669 2915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:46323 (error 108)
I20260924 00:14:18.131911 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3140
I20260924 00:14:18.135380 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:18.132083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:46323 (local address 127.0.0.1:38716)
0924 00:14:18.135303 (+ 3220us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:18.135348 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:46323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3190,"thread_start_us":62,"threads_started":1}
W20260924 00:14:18.135491 2915 catalog_manager.cc:4584] TS b1ce649fabd14a6ba0799c40347487d7 (127.1.44.129:46323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e6dce949bc44daebb1b886f041ae8d4: Network error: Client connection negotiation failed: client connection to 127.1.44.129:46323: connect: Connection refused (error 111)
I20260924 00:14:18.137761 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3280
I20260924 00:14:18.143155 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 2896
2026-09-24T00:14:18Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5886 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:18 dist-test-slave-cm7b krb5kdc[3586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:18 dist-test-slave-cm7b krb5kdc[3586](info): set up 2 sockets
Sep 24 00:14:18 dist-test-slave-cm7b krb5kdc[3586](info): commencing operation
krb5kdc: starting...
W20260924 00:14:19.167089 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:19.277534 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46177
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:46177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:19.355995 3599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:19.356191 3599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:19.356213 3599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:19.357267 3599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:19.357299 3599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:19.357313 3599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:19.357326 3599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:19.357338 3599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:19.358678 3599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:46177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:19.358851 3599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:19.359004 3599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:19.363905 3605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:19.363974 3599 server_base.cc:1034] running on GCE node
W20260924 00:14:19.363931 3607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.363931 3604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:19.364305 3599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:19.364540 3599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:19.365675 3599 hybrid_clock.cc:648] HybridClock initialized: now 1790208859365665 us; error 27 us; skew 500 ppm
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:19.368094 3599 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:19.368553 3599 webserver.cc:460] Webserver started at http://127.1.44.190:41151/ using document root <none> and password file <none>
I20260924 00:14:19.368782 3599 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:19.368827 3599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:19.368992 3599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:19.369992 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "8d387cef8fc849759a1107d65d3b17d7"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "f675e272db192f76a41a6d25342bd83e"
server_key_iv: "48695e69852991da5b7835e36dd14223"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.370345 3599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "8d387cef8fc849759a1107d65d3b17d7"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "f675e272db192f76a41a6d25342bd83e"
server_key_iv: "48695e69852991da5b7835e36dd14223"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.371717 3599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:19.372406 3614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.372577 3599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:19.372668 3599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "8d387cef8fc849759a1107d65d3b17d7"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "f675e272db192f76a41a6d25342bd83e"
server_key_iv: "48695e69852991da5b7835e36dd14223"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.372746 3599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:19.396955 3599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:19.398699 3599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:19.398854 3599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:19.402783 3599 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:46177
I20260924 00:14:19.402813 3666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:46177 every 8 connection(s)
I20260924 00:14:19.403131 3599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:19.404039 3667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:19.406147 3667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7: Bootstrap starting.
I20260924 00:14:19.406651 3667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:19.406880 3667 log.cc:824] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:19.407563 3667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7: No bootstrap required, opened a new log
I20260924 00:14:19.409060 3667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d387cef8fc849759a1107d65d3b17d7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46177 } }
I20260924 00:14:19.409204 3667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:19.409226 3667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d387cef8fc849759a1107d65d3b17d7, State: Initialized, Role: FOLLOWER
I20260924 00:14:19.409358 3667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d387cef8fc849759a1107d65d3b17d7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46177 } }
I20260924 00:14:19.409436 3667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:19.409459 3667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:19.409487 3667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:19.409942 3667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d387cef8fc849759a1107d65d3b17d7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46177 } }
I20260924 00:14:19.410022 3667 leader_election.cc:302] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d387cef8fc849759a1107d65d3b17d7; no voters:
I20260924 00:14:19.410172 3667 leader_election.cc:288] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:19.410203 3670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:19.410434 3667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:19.410511 3670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [term 1 LEADER]: Becoming Leader. State: Replica: 8d387cef8fc849759a1107d65d3b17d7, State: Running, Role: LEADER
I20260924 00:14:19.410673 3670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d387cef8fc849759a1107d65d3b17d7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46177 } }
I20260924 00:14:19.411043 3672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d387cef8fc849759a1107d65d3b17d7. Latest consensus state: current_term: 1 leader_uuid: "8d387cef8fc849759a1107d65d3b17d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d387cef8fc849759a1107d65d3b17d7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46177 } } }
I20260924 00:14:19.411134 3672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:19.411437 3671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d387cef8fc849759a1107d65d3b17d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d387cef8fc849759a1107d65d3b17d7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46177 } } }
I20260924 00:14:19.411564 3671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:19.411619 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3599
I20260924 00:14:19.411718 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:19.411767 3679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:19.411909 1202 external_mini_cluster.cc:1428] Setting key dc5fc858f133055c8e30470f1e01f214
I20260924 00:14:19.412413 3679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:19.413972 3679 catalog_manager.cc:1347] Generated new cluster ID: 50d5f30ec3a64256b116b431166737ed
I20260924 00:14:19.414026 3679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:19.421901 3679 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:19.422459 3679 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:19.423511 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.412357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.1.44.190:46177)
0924 00:14:19.412732 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:19.412740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:19.412761 (+ 21us) server_negotiation.cc:407] Connection header received
0924 00:14:19.413014 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:19.413022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:19.413081 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:19.413237 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:19.414026 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.415476 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.416564 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.417149 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.417249 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.419954 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:19.419959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:19.419959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:19.419970 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:19.421919 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.422307 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.422310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.422310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.422407 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.422629 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.422630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.422631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.422888 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:19.423024 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:19.423154 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 00:14:19.423316 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":75,"threads_started":1}
I20260924 00:14:19.427671 3679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d387cef8fc849759a1107d65d3b17d7: Generated new TSK 0
I20260924 00:14:19.427834 3679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:19.454591 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46177
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:19.558379 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:19.558593 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:19.558630 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:19.559865 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:19.559928 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:19.559974 3693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:19.561395 3693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:19.561645 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:19.561857 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:19.567502 3698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.567509 3699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.567672 3701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:19.567776 3693 server_base.cc:1034] running on GCE node
I20260924 00:14:19.568028 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:19.568285 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:19.569476 3693 hybrid_clock.cc:648] HybridClock initialized: now 1790208859569463 us; error 27 us; skew 500 ppm
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:19.572209 3693 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:19.572717 3693 webserver.cc:460] Webserver started at http://127.1.44.129:37745/ using document root <none> and password file <none>
I20260924 00:14:19.572956 3693 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:19.573026 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:19.573161 3693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:19.574187 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "903ef6992c88404e8b472e8aa22c454d"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "c6340d17795e1bb663144608c36d2c48"
server_key_iv: "eeb4642f25491912e61adc9d9affefd2"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.574532 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "903ef6992c88404e8b472e8aa22c454d"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "c6340d17795e1bb663144608c36d2c48"
server_key_iv: "eeb4642f25491912e61adc9d9affefd2"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.575965 3693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:19.576761 3708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.576960 3693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.577060 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "903ef6992c88404e8b472e8aa22c454d"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "c6340d17795e1bb663144608c36d2c48"
server_key_iv: "eeb4642f25491912e61adc9d9affefd2"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.577157 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:19.593510 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:19.595765 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:19.595912 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:19.596115 3693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:19.596391 3693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:19.596443 3693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.596482 3693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:19.596508 3693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.602169 3693 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:44787
I20260924 00:14:19.602207 3821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:44787 every 8 connection(s)
I20260924 00:14:19.602506 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:19.609501 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3693
I20260924 00:14:19.609573 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:19.609735 1202 external_mini_cluster.cc:1428] Setting key ec1e273d5374319c493e6c22e9470662
I20260924 00:14:19.611662 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.603229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:58479 (local address 127.1.44.190:46177)
0924 00:14:19.603286 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:19.603289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:19.603497 (+ 208us) server_negotiation.cc:407] Connection header received
0924 00:14:19.603890 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:19.603891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:19.603902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:19.603953 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:19.604772 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.605266 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.606169 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.606350 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.606399 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.608434 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:19.608435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:19.608435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:19.608441 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:19.609945 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.610509 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.610510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.610510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.610578 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.610812 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.610814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.610814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.610988 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:19.611058 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:19.611432 (+ 374us) server_negotiation.cc:299] Negotiation successful
0924 00:14:19.611521 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 00:14:19.611794 3824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.603159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46177 (local address 127.1.44.129:58479)
0924 00:14:19.603362 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:19.603393 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:19.603805 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:19.604006 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:19.604009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:19.604522 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:19.604695 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:19.604699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.605342 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.605342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:19.606091 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:19.606093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.606418 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.606419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:19.606507 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.606964 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:19.606979 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:19.608351 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:19.610088 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:19.610094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:19.610096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:19.610395 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:19.610645 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:19.610646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:19.610646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:19.610742 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:19.611052 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:19.611056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:19.611277 (+ 221us) client_negotiation.cc:769] Sending connection context
0924 00:14:19.611405 (+ 128us) client_negotiation.cc:240] Negotiation successful
0924 00:14:19.611609 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":124,"threads_started":1}
I20260924 00:14:19.612377 3822 heartbeater.cc:347] Connected to a master server at 127.1.44.190:46177
I20260924 00:14:19.612468 3822 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:19.612646 3822 heartbeater.cc:511] Master 127.1.44.190:46177 requested a full tablet report, sending...
I20260924 00:14:19.613088 3631 ts_manager.cc:194] Registered new tserver with Master: 903ef6992c88404e8b472e8aa22c454d (127.1.44.129:44787)
I20260924 00:14:19.613680 3631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:58479
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:19.633690 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46177
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:19.714442 3827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:19.714622 3827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:19.714643 3827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:19.715826 3827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:19.715864 3827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:19.715912 3827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:19.717316 3827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:19.717543 3827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:19.717747 3827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:19.722509 3833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.722532 3832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.722549 3835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:19.722566 3827 server_base.cc:1034] running on GCE node
I20260924 00:14:19.722870 3827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:19.723127 3827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:19.724267 3827 hybrid_clock.cc:648] HybridClock initialized: now 1790208859724248 us; error 35 us; skew 500 ppm
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:19.726531 3827 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:19.726974 3827 webserver.cc:460] Webserver started at http://127.1.44.130:42033/ using document root <none> and password file <none>
I20260924 00:14:19.727260 3827 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:19.727316 3827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:19.727455 3827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:19.728415 3827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "7f6f92b6a34c4f0192a74576d3431ea8"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "0b9b5b770800f00fb0c28229145d1846"
server_key_iv: "4ce901fc39e7e1c2cc6294e141635cd2"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.728746 3827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "7f6f92b6a34c4f0192a74576d3431ea8"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "0b9b5b770800f00fb0c28229145d1846"
server_key_iv: "4ce901fc39e7e1c2cc6294e141635cd2"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.729981 3827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:19.730654 3842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.730837 3827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:19.730927 3827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "7f6f92b6a34c4f0192a74576d3431ea8"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "0b9b5b770800f00fb0c28229145d1846"
server_key_iv: "4ce901fc39e7e1c2cc6294e141635cd2"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.730995 3827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:19.743758 3827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:19.745414 3827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:19.745553 3827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:19.745759 3827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:19.746052 3827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:19.746083 3827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.746104 3827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:19.746160 3827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.751791 3827 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:45613
I20260924 00:14:19.751837 3955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:45613 every 8 connection(s)
I20260924 00:14:19.752111 3827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:19.757787 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3827
I20260924 00:14:19.757885 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:19.758090 1202 external_mini_cluster.cc:1428] Setting key 21b1715d222ada259ae8a8033e77326c
I20260924 00:14:19.761108 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.752749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:35617 (local address 127.1.44.190:46177)
0924 00:14:19.752792 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:19.752794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:19.752980 (+ 186us) server_negotiation.cc:407] Connection header received
0924 00:14:19.753391 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:19.753392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:19.753401 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:19.753454 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:19.754263 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.754729 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.755742 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.755919 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.755970 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.758059 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:19.758061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:19.758061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:19.758068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:19.759769 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.760160 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.760162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.760162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.760221 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.760471 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.760472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.760472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.760588 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:19.760664 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:19.760918 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 00:14:19.761020 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:19.761271 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.752702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46177 (local address 127.1.44.130:35617)
0924 00:14:19.752845 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:19.752883 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:19.753311 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:19.753499 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:19.753502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:19.754010 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:19.754190 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:19.754194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.754802 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.754802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:19.755668 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:19.755670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.755985 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.755985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:19.756086 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.756462 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:19.756469 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:19.757930 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:19.759844 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:19.759849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:19.759851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:19.760057 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:19.760296 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:19.760297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:19.760298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:19.760403 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:19.760657 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:19.760662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:19.760792 (+ 130us) client_negotiation.cc:769] Sending connection context
0924 00:14:19.760897 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:14:19.761048 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260924 00:14:19.761844 3956 heartbeater.cc:347] Connected to a master server at 127.1.44.190:46177
I20260924 00:14:19.761914 3956 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:19.762080 3956 heartbeater.cc:511] Master 127.1.44.190:46177 requested a full tablet report, sending...
I20260924 00:14:19.762455 3631 ts_manager.cc:194] Registered new tserver with Master: 7f6f92b6a34c4f0192a74576d3431ea8 (127.1.44.130:45613)
I20260924 00:14:19.762849 3631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:35617
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:19.782665 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46177
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:19.859295 3961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:19.859472 3961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:19.859494 3961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:19.860569 3961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:19.860606 3961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:19.860631 3961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:19.861915 3961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:46177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:19.862078 3961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:19.862233 3961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:19.866859 3969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.866859 3967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:19.866868 3966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:19.867120 3961 server_base.cc:1034] running on GCE node
I20260924 00:14:19.867431 3961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:19.867666 3961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:19.868810 3961 hybrid_clock.cc:648] HybridClock initialized: now 1790208859868799 us; error 25 us; skew 500 ppm
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:19.871322 3961 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:19.871762 3961 webserver.cc:460] Webserver started at http://127.1.44.131:40001/ using document root <none> and password file <none>
I20260924 00:14:19.871980 3961 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:19.872047 3961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:19.872182 3961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:19.873157 3961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "b57a20a2afb044998645403d66efb354"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "d38e8715494da10b37e8c06675893fb3"
server_key_iv: "8c659627ea87291f02c34346d592b321"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.873498 3961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "b57a20a2afb044998645403d66efb354"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "d38e8715494da10b37e8c06675893fb3"
server_key_iv: "8c659627ea87291f02c34346d592b321"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.874711 3961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:19.875563 3976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.875727 3961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.875816 3961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "b57a20a2afb044998645403d66efb354"
format_stamp: "Formatted at 2026-09-24 00:14:19 on dist-test-slave-cm7b"
server_key: "d38e8715494da10b37e8c06675893fb3"
server_key_iv: "8c659627ea87291f02c34346d592b321"
server_key_version: "encryptionkey@0"
I20260924 00:14:19.875900 3961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:19.885155 3961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:19.887290 3961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:19.887439 3961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:19.887656 3961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:19.887939 3961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:19.887990 3961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.888023 3961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:19.888051 3961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:19.893426 3961 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:37813
I20260924 00:14:19.893464 4089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:37813 every 8 connection(s)
I20260924 00:14:19.893757 3961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:19.896574 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 3961
I20260924 00:14:19.896685 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:19.896867 1202 external_mini_cluster.cc:1428] Setting key f9a4ad3f63678b211dc2ea4c5fa31599
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[3586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208859, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:19.902769 3689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.894430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:34525 (local address 127.1.44.190:46177)
0924 00:14:19.894504 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:19.894507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:19.894732 (+ 225us) server_negotiation.cc:407] Connection header received
0924 00:14:19.895164 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:19.895165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:19.895175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:19.895272 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:19.896141 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.896621 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.897986 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.898184 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.898257 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.900366 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:19.900367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:19.900368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:19.900374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:19.901479 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.901856 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.901857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.901858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.901917 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:19.902162 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:19.902163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:19.902164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:19.902270 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:19.902341 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:19.902604 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 00:14:19.902680 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:19.902925 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:19.894395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46177 (local address 127.1.44.131:34525)
0924 00:14:19.894575 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:19.894607 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:19.895081 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:19.895322 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:19.895325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:19.895830 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:19.896066 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:19.896071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.896722 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.896723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:19.897897 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:19.897899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:19.898260 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:19.898261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:19.898389 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:19.898854 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:19.898863 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:19.900264 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:19.901572 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:19.901577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:19.901579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:19.901780 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:19.901985 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:19.901986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:19.901986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:19.902095 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:19.902346 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:19.902351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:19.902475 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 00:14:19.902576 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:14:19.902712 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260924 00:14:19.903419 4090 heartbeater.cc:347] Connected to a master server at 127.1.44.190:46177
I20260924 00:14:19.903508 4090 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:19.903640 4090 heartbeater.cc:511] Master 127.1.44.190:46177 requested a full tablet report, sending...
I20260924 00:14:19.904000 3631 ts_manager.cc:194] Registered new tserver with Master: b57a20a2afb044998645403d66efb354 (127.1.44.131:37813)
I20260924 00:14:19.904369 3631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:34525
I20260924 00:14:19.908484 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 00:14:19.919348 3689 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:19.918590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.1.44.190:46177)
0924 00:14:19.918623 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:19.918624 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:19.918728 (+ 104us) server_negotiation.cc:407] Connection header received
0924 00:14:19.918762 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:19.918762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:19.918770 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:19.918822 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:19.919306 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34056: BlockingRecv error: recv got EOF from 127.0.0.1:34056 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:19.920091 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3693
I20260924 00:14:19.924415 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3827
I20260924 00:14:19.929282 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3961
I20260924 00:14:19.933501 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 3599
2026-09-24T00:14:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1788 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[4102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[4102](info): set up 2 sockets
Sep 24 00:14:19 dist-test-slave-cm7b krb5kdc[4102](info): commencing operation
krb5kdc: starting...
W20260924 00:14:20.962947 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:21 dist-test-slave-cm7b krb5kdc[4102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:21.070359 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36749
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:45887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:36749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:21.148226 4115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:21.148448 4115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:21.148483 4115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:21.149644 4115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:21.149708 4115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:21.149734 4115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:21.149767 4115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:21.149797 4115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:21.151321 4115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:45887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:36749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:21.151571 4115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:21.151787 4115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:21.156401 4121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:21.156492 4123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:21.156528 4115 server_base.cc:1034] running on GCE node
W20260924 00:14:21.156422 4120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:21.156831 4115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:21.157052 4115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:21.158193 4115 hybrid_clock.cc:648] HybridClock initialized: now 1790208861158171 us; error 35 us; skew 500 ppm
Sep 24 00:14:21 dist-test-slave-cm7b krb5kdc[4102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208861, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:21.160516 4115 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:21.160967 4115 webserver.cc:460] Webserver started at http://127.1.44.190:44355/ using document root <none> and password file <none>
I20260924 00:14:21.161198 4115 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:21.161262 4115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:21.161391 4115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:21.162357 4115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "0883b605fec845dfbce880d5dc123fac"
format_stamp: "Formatted at 2026-09-24 00:14:21 on dist-test-slave-cm7b"
server_key: "19960a99dd96dbcbbae7793a2d667a49"
server_key_iv: "d1c4e775fc1778599d5326772798e2ae"
server_key_version: "encryptionkey@0"
I20260924 00:14:21.162703 4115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "0883b605fec845dfbce880d5dc123fac"
format_stamp: "Formatted at 2026-09-24 00:14:21 on dist-test-slave-cm7b"
server_key: "19960a99dd96dbcbbae7793a2d667a49"
server_key_iv: "d1c4e775fc1778599d5326772798e2ae"
server_key_version: "encryptionkey@0"
I20260924 00:14:21.164042 4115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:21.164743 4130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:21.164906 4115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:21.164997 4115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "0883b605fec845dfbce880d5dc123fac"
format_stamp: "Formatted at 2026-09-24 00:14:21 on dist-test-slave-cm7b"
server_key: "19960a99dd96dbcbbae7793a2d667a49"
server_key_iv: "d1c4e775fc1778599d5326772798e2ae"
server_key_version: "encryptionkey@0"
I20260924 00:14:21.165072 4115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:21.185312 4115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:21.186990 4115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:21.187142 4115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:21.190825 4115 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:36749
I20260924 00:14:21.190860 4182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:36749 every 8 connection(s)
I20260924 00:14:21.191210 4115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:21.191967 4183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:21.194085 4183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac: Bootstrap starting.
I20260924 00:14:21.194674 4183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:21.194665 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4115
I20260924 00:14:21.194777 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:21.194950 1202 external_mini_cluster.cc:1428] Setting key 33bc20b3f7bcf1e190cd5310074c5063
I20260924 00:14:21.194988 4183 log.cc:824] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:21.195788 4183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac: No bootstrap required, opened a new log
I20260924 00:14:21.198056 4183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0883b605fec845dfbce880d5dc123fac" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36749 } }
I20260924 00:14:21.198220 4183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:21.198257 4183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0883b605fec845dfbce880d5dc123fac, State: Initialized, Role: FOLLOWER
I20260924 00:14:21.198377 4183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0883b605fec845dfbce880d5dc123fac" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36749 } }
I20260924 00:14:21.198447 4183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:21.198474 4183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:21.198521 4183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:21.199131 4183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0883b605fec845dfbce880d5dc123fac" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36749 } }
I20260924 00:14:21.199273 4183 leader_election.cc:302] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0883b605fec845dfbce880d5dc123fac; no voters:
I20260924 00:14:21.199455 4183 leader_election.cc:288] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:21.199720 4183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:21.199764 4188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:21.199884 4188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [term 1 LEADER]: Becoming Leader. State: Replica: 0883b605fec845dfbce880d5dc123fac, State: Running, Role: LEADER
I20260924 00:14:21.200139 4188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0883b605fec845dfbce880d5dc123fac" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36749 } }
I20260924 00:14:21.200665 4190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0883b605fec845dfbce880d5dc123fac. Latest consensus state: current_term: 1 leader_uuid: "0883b605fec845dfbce880d5dc123fac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0883b605fec845dfbce880d5dc123fac" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36749 } } }
I20260924 00:14:21.200769 4190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:21.201162 4189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0883b605fec845dfbce880d5dc123fac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0883b605fec845dfbce880d5dc123fac" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36749 } } }
I20260924 00:14:21.201395 4201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 00:14:21 dist-test-slave-cm7b krb5kdc[4102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:21.201890 4201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:21.201398 4189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:21.203483 4201 catalog_manager.cc:1347] Generated new cluster ID: fc4fff4ebb6e4c499f7174d9f16db9a9
I20260924 00:14:21.203584 4201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:21.205497 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:21.195428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.1.44.190:36749)
0924 00:14:21.195567 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:21.195575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:21.195610 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:14:21.196057 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:21.196064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:21.196108 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:21.196231 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:21.196994 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:21.197827 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:21.198870 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:21.199391 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:21.199523 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:21.202265 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:21.202269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:21.202270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:21.202281 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:21.204044 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:21.204420 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:21.204423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:21.204423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:21.204512 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:21.204734 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:21.204735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:21.204735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:21.204954 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:21.205138 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:21.205203 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 00:14:21.205332 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260924 00:14:21.206245 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:14:21.223191 4201 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:21.223716 4201 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:21.228760 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:21.224384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.1.44.190:36749)
0924 00:14:21.224422 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:21.224424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:21.224561 (+ 137us) server_negotiation.cc:407] Connection header received
0924 00:14:21.224602 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:21.224603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:21.224613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:21.224701 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:21.225065 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:21.225853 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:21.227152 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:21.227506 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:21.227578 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:21.228296 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:21.228298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:21.228298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:21.228501 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:21.228518 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:21.228525 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:21.228589 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:21.228657 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 00:14:21.228684 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:21.231149 4201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0883b605fec845dfbce880d5dc123fac: Generated new TSK 0
I20260924 00:14:21.231351 4201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:21.234309 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4115
2026-09-24T00:14:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1298 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:21 dist-test-slave-cm7b krb5kdc[4216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:21 dist-test-slave-cm7b krb5kdc[4216](info): set up 2 sockets
Sep 24 00:14:21 dist-test-slave-cm7b krb5kdc[4216](info): commencing operation
krb5kdc: starting...
W20260924 00:14:22.266975 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:22.378078 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34161
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:22.455277 4229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:22.455456 4229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:22.455478 4229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:22.456619 4229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:22.456662 4229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:22.456677 4229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:22.456689 4229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:22.456743 4229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:22.458097 4229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:22.458323 4229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:22.458542 4229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:22.463006 4237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.463052 4235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:22.463181 4229 server_base.cc:1034] running on GCE node
W20260924 00:14:22.463019 4234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:22.463521 4229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:22.463799 4229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:22.464977 4229 hybrid_clock.cc:648] HybridClock initialized: now 1790208862464953 us; error 33 us; skew 500 ppm
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:22.467363 4229 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:22.467767 4229 webserver.cc:460] Webserver started at http://127.1.44.190:39427/ using document root <none> and password file <none>
I20260924 00:14:22.467995 4229 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:22.468062 4229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:22.468199 4229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:22.469175 4229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "7d74f32f9d644ff482f7046f76444d55"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "35f821bbfcc63a20558f38cebe2bff45"
server_key_iv: "e9033208234f8ef6f650df696239c988"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.469530 4229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "7d74f32f9d644ff482f7046f76444d55"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "35f821bbfcc63a20558f38cebe2bff45"
server_key_iv: "e9033208234f8ef6f650df696239c988"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.470749 4229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:14:22.471475 4244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.471657 4229 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:22.471758 4229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "7d74f32f9d644ff482f7046f76444d55"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "35f821bbfcc63a20558f38cebe2bff45"
server_key_iv: "e9033208234f8ef6f650df696239c988"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.471837 4229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:22.492926 4229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:22.495077 4229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:22.495251 4229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:22.498807 4229 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34161
I20260924 00:14:22.498840 4296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34161 every 8 connection(s)
I20260924 00:14:22.499151 4229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:22.500078 4297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:22.502063 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4229
I20260924 00:14:22.502141 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:22.502285 4297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55: Bootstrap starting.
I20260924 00:14:22.502307 1202 external_mini_cluster.cc:1428] Setting key 1fd20b91d6ec100a7fa512e49401d56f
I20260924 00:14:22.502988 4297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:22.503338 4297 log.cc:824] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:22.504031 4297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55: No bootstrap required, opened a new log
I20260924 00:14:22.506253 4297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74f32f9d644ff482f7046f76444d55" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34161 } }
I20260924 00:14:22.506372 4297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:22.506393 4297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d74f32f9d644ff482f7046f76444d55, State: Initialized, Role: FOLLOWER
I20260924 00:14:22.506516 4297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74f32f9d644ff482f7046f76444d55" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34161 } }
I20260924 00:14:22.506597 4297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:22.506644 4297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:22.506691 4297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:22.507375 4297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74f32f9d644ff482f7046f76444d55" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34161 } }
I20260924 00:14:22.507480 4297 leader_election.cc:302] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d74f32f9d644ff482f7046f76444d55; no voters:
I20260924 00:14:22.507632 4297 leader_election.cc:288] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:22.507666 4302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:22.507874 4302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [term 1 LEADER]: Becoming Leader. State: Replica: 7d74f32f9d644ff482f7046f76444d55, State: Running, Role: LEADER
I20260924 00:14:22.507917 4297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:22.508066 4302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74f32f9d644ff482f7046f76444d55" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34161 } }
I20260924 00:14:22.508525 4304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d74f32f9d644ff482f7046f76444d55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74f32f9d644ff482f7046f76444d55" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34161 } } }
I20260924 00:14:22.508754 4304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:22.508682 4303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d74f32f9d644ff482f7046f76444d55. Latest consensus state: current_term: 1 leader_uuid: "7d74f32f9d644ff482f7046f76444d55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74f32f9d644ff482f7046f76444d55" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34161 } } }
I20260924 00:14:22.509306 4303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:22.509456 4314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:22.509943 4314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:22.511507 4314 catalog_manager.cc:1347] Generated new cluster ID: e2ff7fe0983845748af0927b7036614e
I20260924 00:14:22.511559 4314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:22.513897 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.502813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52106 (local address 127.1.44.190:34161)
0924 00:14:22.502998 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:22.503006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:22.503039 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:14:22.503654 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:22.503661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:22.503732 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:22.503874 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:22.504496 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.505916 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.506860 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.507487 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.507600 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.510358 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:22.510362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:22.510362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:22.510373 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:22.512351 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.512762 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.512764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.512765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.512856 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.513071 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.513071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.513071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.513329 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:22.513465 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:22.513593 (+ 128us) server_negotiation.cc:299] Negotiation successful
0924 00:14:22.513709 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
I20260924 00:14:22.519407 4314 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:22.520053 4314 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:22.527913 4314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d74f32f9d644ff482f7046f76444d55: Generated new TSK 0
I20260924 00:14:22.528071 4314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:22.558372 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34161
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:22.635542 4323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:22.635737 4323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:22.635758 4323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:22.636886 4323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:22.636926 4323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:22.636981 4323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:22.638362 4323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:22.638595 4323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:22.638808 4323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:22.643357 4331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.643558 4328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.643356 4329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:22.643690 4323 server_base.cc:1034] running on GCE node
I20260924 00:14:22.643880 4323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:22.644148 4323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:22.645320 4323 hybrid_clock.cc:648] HybridClock initialized: now 1790208862645304 us; error 26 us; skew 500 ppm
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:22.647782 4323 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:22.648191 4323 webserver.cc:460] Webserver started at http://127.1.44.129:37407/ using document root <none> and password file <none>
I20260924 00:14:22.648403 4323 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:22.648468 4323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:22.648615 4323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:22.649586 4323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "0190873073d741fc930268b2e3685a3d"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "9b16ff4048a5afbb3809229ab0a4581b"
server_key_iv: "c008cb56e490c5211112d2f47841572c"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.649935 4323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "0190873073d741fc930268b2e3685a3d"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "9b16ff4048a5afbb3809229ab0a4581b"
server_key_iv: "c008cb56e490c5211112d2f47841572c"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.651175 4323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:22.651957 4338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.652141 4323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:22.652237 4323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "0190873073d741fc930268b2e3685a3d"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "9b16ff4048a5afbb3809229ab0a4581b"
server_key_iv: "c008cb56e490c5211112d2f47841572c"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.652310 4323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:22.660924 4323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:22.663149 4323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:22.663321 4323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:22.663527 4323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:22.663798 4323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:22.663851 4323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.663889 4323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:22.663918 4323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.669466 4323 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:36907
I20260924 00:14:22.669528 4451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:36907 every 8 connection(s)
I20260924 00:14:22.669821 4323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:22.672617 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4323
I20260924 00:14:22.672729 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:22.672901 1202 external_mini_cluster.cc:1428] Setting key b13cd56a628f8591122308b09a8e7231
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:22.679478 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.670451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:52701 (local address 127.1.44.190:34161)
0924 00:14:22.670489 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:22.670491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:22.670681 (+ 190us) server_negotiation.cc:407] Connection header received
0924 00:14:22.671157 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:22.671158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:22.671171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:22.671255 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:22.672139 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.672640 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.673862 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.674089 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.674174 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.676650 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:22.676651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:22.676652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:22.676659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:22.678019 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.678532 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.678533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.678533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.678581 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.678862 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.678863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.678863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.678954 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:22.679002 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:22.679348 (+ 346us) server_negotiation.cc:299] Negotiation successful
0924 00:14:22.679417 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:22.679721 4454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.670423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34161 (local address 127.1.44.129:52701)
0924 00:14:22.670547 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:22.670610 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:22.671055 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:22.671329 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:22.671333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:22.671862 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:22.672038 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:22.672042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.672742 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.672744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:22.673778 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:22.673781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.674185 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.674186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:22.674328 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.674829 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:22.674838 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:22.676560 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:22.678103 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:22.678111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:22.678114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:22.678438 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:22.678648 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:22.678649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:22.678649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:22.678797 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:22.679019 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:22.679025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:22.679184 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 00:14:22.679331 (+ 147us) client_negotiation.cc:240] Negotiation successful
0924 00:14:22.679497 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260924 00:14:22.680258 4452 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34161
I20260924 00:14:22.680359 4452 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:22.680548 4452 heartbeater.cc:511] Master 127.1.44.190:34161 requested a full tablet report, sending...
I20260924 00:14:22.681083 4261 ts_manager.cc:194] Registered new tserver with Master: 0190873073d741fc930268b2e3685a3d (127.1.44.129:36907)
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:14:22.681895 4261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:52701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:22.696084 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34161
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:22.773244 4457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:22.773424 4457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:22.773447 4457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:22.774693 4457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:22.774731 4457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:22.774757 4457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:22.776240 4457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:22.776471 4457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:22.776691 4457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:22.781359 4463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.781365 4462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.781441 4465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:22.781571 4457 server_base.cc:1034] running on GCE node
I20260924 00:14:22.781953 4457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:22.782191 4457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:22.783320 4457 hybrid_clock.cc:648] HybridClock initialized: now 1790208862783302 us; error 33 us; skew 500 ppm
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:22.785611 4457 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:22.786064 4457 webserver.cc:460] Webserver started at http://127.1.44.130:41201/ using document root <none> and password file <none>
I20260924 00:14:22.786280 4457 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:22.786345 4457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:22.786486 4457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:22.787510 4457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "cf1268e2276747509de1095ee8f29167"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "96a5a46c4a80758b8262d5b8d84c674d"
server_key_iv: "0d2562a243ddf1183b7161ee6a9b2746"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.787864 4457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "cf1268e2276747509de1095ee8f29167"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "96a5a46c4a80758b8262d5b8d84c674d"
server_key_iv: "0d2562a243ddf1183b7161ee6a9b2746"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.789135 4457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:22.789820 4472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.789988 4457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:22.790081 4457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "cf1268e2276747509de1095ee8f29167"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "96a5a46c4a80758b8262d5b8d84c674d"
server_key_iv: "0d2562a243ddf1183b7161ee6a9b2746"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.790162 4457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:22.802590 4457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:22.804312 4457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:22.804469 4457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:22.804697 4457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:22.804987 4457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:22.805039 4457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.805073 4457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:22.805099 4457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.810830 4457 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:40721
I20260924 00:14:22.810892 4585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:40721 every 8 connection(s)
I20260924 00:14:22.811206 4457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:22.819159 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.811926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48511 (local address 127.1.44.190:34161)
0924 00:14:22.811996 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:22.811998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:22.812150 (+ 152us) server_negotiation.cc:407] Connection header received
0924 00:14:22.812566 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:22.812567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:22.812576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:22.812626 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:22.813423 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.813901 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.814800 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.814976 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.815024 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.816983 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:22.816984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:22.816985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:22.816991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:22.818114 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.818442 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.818443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.818443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.818490 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.818696 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.818697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.818697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.818788 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:22.818837 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:22.819037 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 00:14:22.819099 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:22.819365 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.811875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34161 (local address 127.1.44.130:48511)
0924 00:14:22.812042 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:22.812073 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:22.812481 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:22.812674 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:22.812677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:22.813189 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:22.813354 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:22.813358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.813973 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.813973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:22.814732 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:22.814734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.815044 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.815045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:22.815137 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.815641 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:22.815650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:22.816906 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:22.818183 (+ 1277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:22.818188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:22.818189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:22.818379 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:22.818551 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:22.818552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:22.818552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:22.818643 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:22.818841 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:22.818844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:22.818938 (+ 94us) client_negotiation.cc:769] Sending connection context
0924 00:14:22.819016 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 00:14:22.819121 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260924 00:14:22.819844 4586 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34161
I20260924 00:14:22.819947 4586 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:22.819974 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4457
I20260924 00:14:22.820070 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:22.820145 4586 heartbeater.cc:511] Master 127.1.44.190:34161 requested a full tablet report, sending...
I20260924 00:14:22.820264 1202 external_mini_cluster.cc:1428] Setting key bc8f8e4660aa5fa1a848ff92f2664d67
I20260924 00:14:22.820544 4261 ts_manager.cc:194] Registered new tserver with Master: cf1268e2276747509de1095ee8f29167 (127.1.44.130:40721)
I20260924 00:14:22.821087 4261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:48511
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:22.843670 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34161
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:22.924424 4591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:22.924607 4591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:22.924629 4591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:22.926046 4591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:22.926107 4591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:22.926164 4591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:22.927738 4591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:22.927910 4591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:22.928072 4591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:22.932732 4596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.932778 4599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:22.932881 4597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:22.932767 4591 server_base.cc:1034] running on GCE node
I20260924 00:14:22.933122 4591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:22.933362 4591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:22.934502 4591 hybrid_clock.cc:648] HybridClock initialized: now 1790208862934495 us; error 38 us; skew 500 ppm
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:22.936932 4591 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:22.937369 4591 webserver.cc:460] Webserver started at http://127.1.44.131:38241/ using document root <none> and password file <none>
I20260924 00:14:22.937584 4591 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:22.937651 4591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:22.937793 4591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:22.938742 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "57e670bb234d4523972f8f69ec864553"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "1578f22256cf322087873db629773d65"
server_key_iv: "312f3b23cf091ceb907cb58855a24514"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.939096 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "57e670bb234d4523972f8f69ec864553"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "1578f22256cf322087873db629773d65"
server_key_iv: "312f3b23cf091ceb907cb58855a24514"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.940392 4591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:22.941145 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.941320 4591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:22.941417 4591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "57e670bb234d4523972f8f69ec864553"
format_stamp: "Formatted at 2026-09-24 00:14:22 on dist-test-slave-cm7b"
server_key: "1578f22256cf322087873db629773d65"
server_key_iv: "312f3b23cf091ceb907cb58855a24514"
server_key_version: "encryptionkey@0"
I20260924 00:14:22.941505 4591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:22.957463 4591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:22.959651 4591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:22.959806 4591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:22.960016 4591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:22.960302 4591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:22.960355 4591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.960395 4591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:22.960423 4591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:22.965960 4591 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:45041
I20260924 00:14:22.965993 4719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:45041 every 8 connection(s)
I20260924 00:14:22.966277 4591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:22.970068 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4591
I20260924 00:14:22.970175 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:22.970361 1202 external_mini_cluster.cc:1428] Setting key 3f52d8087ce5180aadad179c035d174f
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:22.975085 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.966931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:38195 (local address 127.1.44.190:34161)
0924 00:14:22.966973 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:22.966975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:22.967211 (+ 236us) server_negotiation.cc:407] Connection header received
0924 00:14:22.967710 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:22.967711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:22.967721 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:22.967774 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:22.968584 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.969074 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.970008 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.970310 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.970389 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.972532 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:22.972534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:22.972534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:22.972541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:22.973650 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.974110 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.974111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.974112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.974171 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:22.974438 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:22.974438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:22.974439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:22.974550 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:22.974622 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:22.974895 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 00:14:22.974996 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:14:22.975245 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.966911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34161 (local address 127.1.44.131:38195)
0924 00:14:22.967042 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:22.967072 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:22.967612 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:22.967821 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:22.967824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:22.968335 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:22.968510 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:22.968514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.969153 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.969154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:22.969917 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:22.969919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.970392 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.970393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:22.970544 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:22.971013 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:22.971023 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:22.972438 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:22.973746 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:22.973752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:22.973754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:22.974040 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:22.974238 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:22.974239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:22.974239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:22.974383 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:22.974626 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:22.974632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:22.974761 (+ 129us) client_negotiation.cc:769] Sending connection context
0924 00:14:22.974859 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 00:14:22.975001 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":79,"threads_started":1}
I20260924 00:14:22.975678 4720 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34161
I20260924 00:14:22.975749 4720 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:22.975867 4720 heartbeater.cc:511] Master 127.1.44.190:34161 requested a full tablet report, sending...
I20260924 00:14:22.976169 4261 ts_manager.cc:194] Registered new tserver with Master: 57e670bb234d4523972f8f69ec864553 (127.1.44.131:45041)
I20260924 00:14:22.976536 4261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:38195
I20260924 00:14:22.982092 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 00:14:22 dist-test-slave-cm7b krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208862, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:23.003772 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:22.996979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.1.44.190:34161)
0924 00:14:22.997035 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:22.997036 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:22.997141 (+ 105us) server_negotiation.cc:407] Connection header received
0924 00:14:22.997166 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:22.997167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:22.997175 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:22.997226 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:22.997738 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.998188 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.998940 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:22.999159 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:22.999301 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:23.001451 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:23.001452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:23.001452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:23.001458 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:23.002728 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:23.003037 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:23.003038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:23.003038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:23.003086 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:23.003290 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:23.003291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:23.003291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:23.003419 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:23.003504 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:23.003655 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 00:14:23.003709 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260924 00:14:23.004027 4261 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52112
I20260924 00:14:23.004454 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4323
I20260924 00:14:23.009370 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4457
I20260924 00:14:23.013600 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4591
I20260924 00:14:23.017549 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4229
2026-09-24T00:14:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1785 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:23 dist-test-slave-cm7b krb5kdc[4732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:23 dist-test-slave-cm7b krb5kdc[4732](info): set up 2 sockets
Sep 24 00:14:23 dist-test-slave-cm7b krb5kdc[4732](info): commencing operation
krb5kdc: starting...
W20260924 00:14:24.046952 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:24.156006 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37587
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:24.232960 4745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:24.233139 4745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:24.233160 4745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:24.234274 4745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:24.234315 4745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:24.234330 4745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:24.234342 4745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:24.234355 4745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:24.235813 4745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:24.236027 4745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:24.236256 4745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:24.240803 4753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:24.240953 4751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.241014 4745 server_base.cc:1034] running on GCE node
W20260924 00:14:24.240980 4750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.241328 4745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:24.241614 4745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:24.242774 4745 hybrid_clock.cc:648] HybridClock initialized: now 1790208864242771 us; error 37 us; skew 500 ppm
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:24.245203 4745 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:24.245595 4745 webserver.cc:460] Webserver started at http://127.1.44.190:35819/ using document root <none> and password file <none>
I20260924 00:14:24.245802 4745 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:24.245867 4745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:24.246016 4745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:24.246969 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "0f4415716e1c44c5b3774ade3742b55e"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "e100feed04692d2ed78837bfcbc70fd5"
server_key_iv: "ee42d350cc0400de88f26dbe1c17e958"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.247346 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "0f4415716e1c44c5b3774ade3742b55e"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "e100feed04692d2ed78837bfcbc70fd5"
server_key_iv: "ee42d350cc0400de88f26dbe1c17e958"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.248576 4745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:24.249308 4760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.249503 4745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:24.249589 4745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "0f4415716e1c44c5b3774ade3742b55e"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "e100feed04692d2ed78837bfcbc70fd5"
server_key_iv: "ee42d350cc0400de88f26dbe1c17e958"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.249639 4745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:24.260409 4745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:24.262066 4745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:24.262215 4745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:24.265726 4745 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:37587
I20260924 00:14:24.265777 4812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:37587 every 8 connection(s)
I20260924 00:14:24.266187 4745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:24.266891 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:24.268921 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e: Bootstrap starting.
I20260924 00:14:24.269409 4813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:24.269636 4813 log.cc:824] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:24.269835 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4745
I20260924 00:14:24.269959 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:24.270128 1202 external_mini_cluster.cc:1428] Setting key cb2ad4c72e430704fda21d95e1ed25ff
I20260924 00:14:24.270346 4813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e: No bootstrap required, opened a new log
I20260924 00:14:24.272986 4813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4415716e1c44c5b3774ade3742b55e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37587 } }
I20260924 00:14:24.273137 4813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:24.273166 4813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f4415716e1c44c5b3774ade3742b55e, State: Initialized, Role: FOLLOWER
I20260924 00:14:24.273291 4813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4415716e1c44c5b3774ade3742b55e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37587 } }
I20260924 00:14:24.273360 4813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:24.273387 4813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:24.273420 4813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:24.274013 4813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4415716e1c44c5b3774ade3742b55e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37587 } }
I20260924 00:14:24.274093 4813 leader_election.cc:302] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f4415716e1c44c5b3774ade3742b55e; no voters:
I20260924 00:14:24.274281 4813 leader_election.cc:288] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:24.274307 4818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:24.274410 4818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [term 1 LEADER]: Becoming Leader. State: Replica: 0f4415716e1c44c5b3774ade3742b55e, State: Running, Role: LEADER
I20260924 00:14:24.274510 4813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:24.274492 4818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4415716e1c44c5b3774ade3742b55e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37587 } }
I20260924 00:14:24.274781 4820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f4415716e1c44c5b3774ade3742b55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4415716e1c44c5b3774ade3742b55e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37587 } } }
I20260924 00:14:24.274863 4820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:24.275096 4822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:24.275091 4819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f4415716e1c44c5b3774ade3742b55e. Latest consensus state: current_term: 1 leader_uuid: "0f4415716e1c44c5b3774ade3742b55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4415716e1c44c5b3774ade3742b55e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37587 } } }
I20260924 00:14:24.275327 4819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:24.275677 4822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:24.277299 4822 catalog_manager.cc:1347] Generated new cluster ID: 5f5ebb1ccee54af0909abd5b1b163983
I20260924 00:14:24.277854 4822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:24.281265 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.270568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.1.44.190:37587)
0924 00:14:24.270737 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:24.270745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:24.270776 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:14:24.271634 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:24.271642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:24.271701 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:24.271846 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:24.272422 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.273754 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.274754 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.275399 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.275510 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.278171 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:24.278175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:24.278175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:24.278186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:24.279819 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.280179 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.280182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.280182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.280267 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.280470 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.280471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.280471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.280687 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:24.280839 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:24.280940 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 00:14:24.281067 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260924 00:14:24.290202 4822 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:24.290832 4822 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:24.294381 4822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f4415716e1c44c5b3774ade3742b55e: Generated new TSK 0
I20260924 00:14:24.294525 4822 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:24.328230 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37587
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:24.407243 4839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:24.407429 4839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:24.407451 4839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:24.408579 4839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:24.408617 4839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:24.408643 4839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:24.410027 4839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:24.410197 4839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:24.410352 4839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:24.414951 4844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.415032 4839 server_base.cc:1034] running on GCE node
W20260924 00:14:24.414981 4845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:24.414933 4847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.415350 4839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:24.415606 4839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:24.416744 4839 hybrid_clock.cc:648] HybridClock initialized: now 1790208864416727 us; error 36 us; skew 500 ppm
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:24.423062 4839 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:24.423527 4839 webserver.cc:460] Webserver started at http://127.1.44.129:35825/ using document root <none> and password file <none>
I20260924 00:14:24.423748 4839 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:24.423792 4839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:24.423933 4839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:24.424923 4839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "0c3f6999762d4b7a9e6c1341519ddc8e"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "205cedeec9689c630bad81f5b99aab58"
server_key_iv: "60ff2c921a279568923e9451a18d0750"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.425269 4839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "0c3f6999762d4b7a9e6c1341519ddc8e"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "205cedeec9689c630bad81f5b99aab58"
server_key_iv: "60ff2c921a279568923e9451a18d0750"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.426533 4839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:24.427307 4854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.427489 4839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:24.427557 4839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "0c3f6999762d4b7a9e6c1341519ddc8e"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "205cedeec9689c630bad81f5b99aab58"
server_key_iv: "60ff2c921a279568923e9451a18d0750"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.427650 4839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:24.459030 4839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:24.461970 4839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:24.462154 4839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:24.462417 4839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:24.462723 4839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:24.462776 4839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.462818 4839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:24.462845 4839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.469748 4839 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:33811
I20260924 00:14:24.469797 4967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:33811 every 8 connection(s)
I20260924 00:14:24.470129 4839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:24.472486 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4839
I20260924 00:14:24.472615 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:24.472829 1202 external_mini_cluster.cc:1428] Setting key 0a76c7c4e342b6492187abdf93b08172
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:14:24.482607 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.471243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:49941 (local address 127.1.44.190:37587)
0924 00:14:24.471342 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:24.471344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:24.471354 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:24.472757 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:24.472759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:24.472774 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:24.472856 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:24.473781 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.474581 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.476241 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.476474 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.476552 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.479116 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:24.479118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:24.479118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:24.479126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:24.480889 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.481427 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.481428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.481429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.481487 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.481805 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.481807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.481807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.481952 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:24.482043 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:24.482408 (+ 365us) server_negotiation.cc:299] Negotiation successful
0924 00:14:24.482500 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:24.482818 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.470946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37587 (local address 127.1.44.129:49941)
0924 00:14:24.471135 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:24.471183 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:24.472587 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:24.473027 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:24.473032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:24.473390 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:24.473671 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:24.473678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.474742 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.474743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:24.476148 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:24.476150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.476559 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.476560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:24.476689 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.477184 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:24.477201 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:24.479008 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:24.480964 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:24.480971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:24.480973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:24.481310 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:24.481571 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:24.481572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:24.481572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:24.481723 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:24.482036 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:24.482043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:24.482221 (+ 178us) client_negotiation.cc:769] Sending connection context
0924 00:14:24.482376 (+ 155us) client_negotiation.cc:240] Negotiation successful
0924 00:14:24.482556 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260924 00:14:24.483335 4968 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37587
I20260924 00:14:24.483440 4968 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:24.483642 4968 heartbeater.cc:511] Master 127.1.44.190:37587 requested a full tablet report, sending...
I20260924 00:14:24.484200 4777 ts_manager.cc:194] Registered new tserver with Master: 0c3f6999762d4b7a9e6c1341519ddc8e (127.1.44.129:33811)
I20260924 00:14:24.485075 4777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:49941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:24.499485 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37587
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:24.578841 4973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:24.579023 4973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:24.579046 4973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:24.580193 4973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:24.580235 4973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:24.580282 4973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:24.581735 4973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:24.581960 4973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:24.582175 4973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:24.586871 4981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.587003 4973 server_base.cc:1034] running on GCE node
W20260924 00:14:24.586886 4979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:24.586879 4978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.587323 4973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:24.587567 4973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:24.588752 4973 hybrid_clock.cc:648] HybridClock initialized: now 1790208864588739 us; error 41 us; skew 500 ppm
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:24.591013 4973 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:24.591460 4973 webserver.cc:460] Webserver started at http://127.1.44.130:38055/ using document root <none> and password file <none>
I20260924 00:14:24.591688 4973 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:24.591753 4973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:24.591877 4973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:24.592809 4973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "22b423a800b54b6b98039335b9efb18b"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "3f0278a8e7eea4d5d590622bd0b5e972"
server_key_iv: "ddb97039772d36d0f93cc2bc45f70277"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.593159 4973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "22b423a800b54b6b98039335b9efb18b"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "3f0278a8e7eea4d5d590622bd0b5e972"
server_key_iv: "ddb97039772d36d0f93cc2bc45f70277"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.594558 4973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:24.595326 4988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.595520 4973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:24.595609 4973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "22b423a800b54b6b98039335b9efb18b"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "3f0278a8e7eea4d5d590622bd0b5e972"
server_key_iv: "ddb97039772d36d0f93cc2bc45f70277"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.595690 4973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:24.610649 4973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:24.612565 4973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:24.612708 4973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:24.612901 4973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:24.613168 4973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:24.613219 4973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.613260 4973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:24.613284 4973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.618785 4973 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:34019
I20260924 00:14:24.618819 5101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:34019 every 8 connection(s)
I20260924 00:14:24.619115 4973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:24.623574 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 4973
I20260924 00:14:24.623696 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:24.623874 1202 external_mini_cluster.cc:1428] Setting key 15285282cdc48effffba4801fa9fc358
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:24.627941 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.619777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:46189 (local address 127.1.44.190:37587)
0924 00:14:24.619816 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:24.619818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:24.620029 (+ 211us) server_negotiation.cc:407] Connection header received
0924 00:14:24.620848 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:24.620849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:24.620860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:24.620919 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:24.621435 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.621901 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.622817 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.623000 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.623049 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.625147 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:24.625149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:24.625149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:24.625156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:24.626522 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.626999 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.627000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.627000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.627048 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.627321 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.627322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.627322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.627419 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:24.627477 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:24.627798 (+ 321us) server_negotiation.cc:299] Negotiation successful
0924 00:14:24.627857 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:24.628207 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.619738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37587 (local address 127.1.44.130:46189)
0924 00:14:24.619886 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:24.619924 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:24.620765 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:24.620973 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:24.620976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:24.621200 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:24.621367 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:24.621370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.621997 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.621997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:24.622748 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:24.622750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.623065 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.623066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:24.623156 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.623682 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:24.623692 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:24.625061 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:24.626607 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:24.626615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:24.626618 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:24.626924 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:24.627110 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:24.627111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:24.627112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:24.627264 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:24.627495 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:24.627500 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:24.627658 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 00:14:24.627773 (+ 115us) client_negotiation.cc:240] Negotiation successful
0924 00:14:24.627936 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260924 00:14:24.628616 5102 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37587
I20260924 00:14:24.628711 5102 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:24.628885 5102 heartbeater.cc:511] Master 127.1.44.190:37587 requested a full tablet report, sending...
I20260924 00:14:24.629254 4777 ts_manager.cc:194] Registered new tserver with Master: 22b423a800b54b6b98039335b9efb18b (127.1.44.130:34019)
I20260924 00:14:24.629623 4777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:46189
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:24.647773 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37587
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:24.724259 5107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:24.724472 5107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:24.724506 5107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:24.725605 5107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:24.725668 5107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:24.725710 5107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:24.727159 5107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:24.727415 5107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:24.727626 5107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:24.732492 5115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:24.732612 5113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.732613 5107 server_base.cc:1034] running on GCE node
W20260924 00:14:24.732690 5112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:24.732982 5107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:24.733163 5107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:24.734293 5107 hybrid_clock.cc:648] HybridClock initialized: now 1790208864734286 us; error 29 us; skew 500 ppm
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:24.736701 5107 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:24.737125 5107 webserver.cc:460] Webserver started at http://127.1.44.131:33297/ using document root <none> and password file <none>
I20260924 00:14:24.737288 5107 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:24.737327 5107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:24.737408 5107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:24.738287 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "eb28788de92240adbc40f577ca4896cc"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "2633c4162cc5a219e28f25592322e85a"
server_key_iv: "32ea4f9a0887a78f1d2250e755d1d002"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.738569 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "eb28788de92240adbc40f577ca4896cc"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "2633c4162cc5a219e28f25592322e85a"
server_key_iv: "32ea4f9a0887a78f1d2250e755d1d002"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.739822 5107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:24.740473 5122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.740655 5107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:24.740722 5107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "eb28788de92240adbc40f577ca4896cc"
format_stamp: "Formatted at 2026-09-24 00:14:24 on dist-test-slave-cm7b"
server_key: "2633c4162cc5a219e28f25592322e85a"
server_key_iv: "32ea4f9a0887a78f1d2250e755d1d002"
server_key_version: "encryptionkey@0"
I20260924 00:14:24.740795 5107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:24.761031 5107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:24.763247 5107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:24.763396 5107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:24.763607 5107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:24.763878 5107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:24.763911 5107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.763931 5107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:24.763945 5107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:24.769594 5107 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:41725
I20260924 00:14:24.769640 5235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:41725 every 8 connection(s)
I20260924 00:14:24.769901 5107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:24.772059 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5107
I20260924 00:14:24.772177 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:24.772387 1202 external_mini_cluster.cc:1428] Setting key 0c19ee3c06ef8833c8a50f730908c270
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:24.778808 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.770496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:35583 (local address 127.1.44.190:37587)
0924 00:14:24.770567 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:24.770570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:24.770755 (+ 185us) server_negotiation.cc:407] Connection header received
0924 00:14:24.771726 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:24.771728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:24.771739 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:24.771785 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:24.772378 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.773138 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.774150 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.774427 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.774485 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.776666 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:24.776667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:24.776667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:24.776674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:24.777775 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.778117 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.778117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.778118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.778161 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.778359 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.778359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.778359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.778446 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:24.778494 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:24.778697 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 00:14:24.778750 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:14:24.779007 5238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.770487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37587 (local address 127.1.44.131:35583)
0924 00:14:24.770640 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:24.770672 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:24.771636 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:24.771839 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:24.771842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:24.772068 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:24.772292 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:24.772296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.773230 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.773231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:24.774060 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:24.774062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.774499 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.774499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:24.774588 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.775078 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:24.775085 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:24.776592 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:24.777844 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:24.777849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:24.777851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:24.778055 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:24.778217 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:24.778218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:24.778218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:24.778307 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:24.778501 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:24.778504 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:24.778598 (+ 94us) client_negotiation.cc:769] Sending connection context
0924 00:14:24.778672 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:14:24.778776 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":101,"threads_started":1}
I20260924 00:14:24.779542 5236 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37587
I20260924 00:14:24.779630 5236 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:24.779805 5236 heartbeater.cc:511] Master 127.1.44.190:37587 requested a full tablet report, sending...
I20260924 00:14:24.780148 4777 ts_manager.cc:194] Registered new tserver with Master: eb28788de92240adbc40f577ca4896cc (127.1.44.131:41725)
I20260924 00:14:24.780467 4777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:35583
I20260924 00:14:24.783979 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:24.805835 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.798219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.1.44.129:33811)
0924 00:14:24.798375 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:24.798378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:24.798417 (+ 39us) server_negotiation.cc:407] Connection header received
0924 00:14:24.798441 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:24.798442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:24.798454 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:24.798537 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:24.799120 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.799886 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.800649 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.800957 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.801035 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.803156 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:24.803162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:24.803162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:24.803168 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:24.804811 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.805140 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.805142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.805143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.805194 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.805363 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.805364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.805364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.805480 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:24.805576 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:24.805648 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 00:14:24.805704 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[4732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:24.813162 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:24.807025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.1.44.190:37587)
0924 00:14:24.807063 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:24.807065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:24.807098 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:14:24.807148 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:24.807148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:24.807156 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:24.807240 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:24.807646 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.808088 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.808698 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:24.808864 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:24.808920 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:24.810854 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:24.810855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:24.810855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:24.810865 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:24.812280 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.812565 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.812565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.812566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.812611 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:24.812750 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:24.812750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:24.812750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:24.812861 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:24.812927 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:24.813014 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 00:14:24.813070 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 00:14:24.813412 4777 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51950
I20260924 00:14:24.813743 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4839
I20260924 00:14:24.817826 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4973
I20260924 00:14:24.821887 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5107
I20260924 00:14:24.825868 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 4745
2026-09-24T00:14:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1808 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[5248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[5248](info): set up 2 sockets
Sep 24 00:14:24 dist-test-slave-cm7b krb5kdc[5248](info): commencing operation
krb5kdc: starting...
W20260924 00:14:25.859020 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:25 dist-test-slave-cm7b krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:25.972213 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36527
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:36527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:26.049875 5261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:26.050105 5261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:26.050144 5261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:26.051311 5261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:26.051374 5261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:26.051398 5261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:26.051417 5261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:26.051447 5261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:26.052889 5261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:36527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:26.053149 5261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:26.053363 5261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:26.058135 5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:26.058113 5267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.058301 5261 server_base.cc:1034] running on GCE node
W20260924 00:14:26.058351 5266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.058562 5261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:26.058804 5261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:26.059964 5261 hybrid_clock.cc:648] HybridClock initialized: now 1790208866059960 us; error 35 us; skew 500 ppm
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:26.062359 5261 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:26.062718 5261 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:26.062788 5261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:26.062927 5261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:26.063987 5261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "f9640d19265145c198d253461480ba61"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "ca140ab7cdf6008daa8bad66ba4d3ea2"
server_key_iv: "488727d0664e427e7f743371c7ce44ee"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.064342 5261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "f9640d19265145c198d253461480ba61"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "ca140ab7cdf6008daa8bad66ba4d3ea2"
server_key_iv: "488727d0664e427e7f743371c7ce44ee"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.065553 5261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:14:26.066238 5275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.066406 5261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:26.066501 5261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "f9640d19265145c198d253461480ba61"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "ca140ab7cdf6008daa8bad66ba4d3ea2"
server_key_iv: "488727d0664e427e7f743371c7ce44ee"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.066581 5261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:26.085572 5261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:26.087738 5261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:26.087895 5261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:26.091605 5261 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:36527
I20260924 00:14:26.091635 5327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:36527 every 8 connection(s)
I20260924 00:14:26.091902 5261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:26.092739 5328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:26.094782 5328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61: Bootstrap starting.
I20260924 00:14:26.095427 5328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:26.095711 5328 log.cc:824] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:26.096235 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5261
I20260924 00:14:26.096338 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:26.096411 5328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61: No bootstrap required, opened a new log
I20260924 00:14:26.096550 1202 external_mini_cluster.cc:1428] Setting key e03e209de7dc2aa780a1874c90671488
I20260924 00:14:26.099139 5328 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9640d19265145c198d253461480ba61" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36527 } }
I20260924 00:14:26.099314 5328 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:26.099341 5328 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9640d19265145c198d253461480ba61, State: Initialized, Role: FOLLOWER
I20260924 00:14:26.099460 5328 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9640d19265145c198d253461480ba61" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36527 } }
I20260924 00:14:26.099537 5328 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:26.099560 5328 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:26.099624 5328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:26.100116 5328 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9640d19265145c198d253461480ba61" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36527 } }
I20260924 00:14:26.100195 5328 leader_election.cc:302] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9640d19265145c198d253461480ba61; no voters:
I20260924 00:14:26.100376 5328 leader_election.cc:288] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:26.100400 5333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:26.100517 5333 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [term 1 LEADER]: Becoming Leader. State: Replica: f9640d19265145c198d253461480ba61, State: Running, Role: LEADER
I20260924 00:14:26.100618 5328 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:26.100598 5333 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9640d19265145c198d253461480ba61" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36527 } }
I20260924 00:14:26.100927 5333 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9640d19265145c198d253461480ba61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9640d19265145c198d253461480ba61" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36527 } } }
I20260924 00:14:26.100997 5333 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:26.101123 5333 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9640d19265145c198d253461480ba61. Latest consensus state: current_term: 1 leader_uuid: "f9640d19265145c198d253461480ba61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9640d19265145c198d253461480ba61" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36527 } } }
I20260924 00:14:26.101191 5333 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:26.101382 5338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:26.101879 5338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:26.103366 5338 catalog_manager.cc:1347] Generated new cluster ID: 1021723484834d4ebdccb96a9515d789
I20260924 00:14:26.103417 5338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:26.106961 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.097009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.1.44.190:36527)
0924 00:14:26.097189 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.097204 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.097227 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:14:26.097760 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.097766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.097809 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.097913 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:26.098651 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.099656 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.100523 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.101145 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.101249 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.104013 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:26.104016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:26.104017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:26.104034 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:26.105643 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.105950 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.105953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.105954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.106051 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.106219 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.106219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.106220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.106473 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:26.106591 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.106667 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.106779 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260924 00:14:26.123060 5338 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:26.123602 5338 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:26.131897 5338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61: Generated new TSK 0
I20260924 00:14:26.132076 5338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:26.162318 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36527
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:26.244681 5354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:26.244895 5354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:26.244920 5354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:26.246030 5354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:26.246065 5354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:26.246090 5354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:26.247648 5354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:26.247843 5354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:26.248019 5354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:26.252974 5362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:26.252974 5359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.253155 5354 server_base.cc:1034] running on GCE node
W20260924 00:14:26.253007 5360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.253427 5354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:26.253674 5354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:26.254818 5354 hybrid_clock.cc:648] HybridClock initialized: now 1790208866254805 us; error 29 us; skew 500 ppm
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:26.257383 5354 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:26.257731 5354 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:26.257777 5354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:26.257911 5354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:26.258961 5354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "2cc7c4b5c306d8a895f30eb5eee91e57"
server_key_iv: "1d46f8ec8ffddbc1b12f73ef162061d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.259356 5354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "2cc7c4b5c306d8a895f30eb5eee91e57"
server_key_iv: "1d46f8ec8ffddbc1b12f73ef162061d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.260649 5354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:26.261363 5368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.261588 5354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:26.261664 5354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "2cc7c4b5c306d8a895f30eb5eee91e57"
server_key_iv: "1d46f8ec8ffddbc1b12f73ef162061d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.261754 5354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:26.277141 5354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:26.279013 5354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:26.279181 5354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:26.279523 5354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:26.279843 5354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:26.279877 5354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.279901 5354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:26.279946 5354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.285506 5354 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:37161
I20260924 00:14:26.285584 5481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:37161 every 8 connection(s)
I20260924 00:14:26.285854 5354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:26.287371 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5354
I20260924 00:14:26.287494 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:26.287679 1202 external_mini_cluster.cc:1428] Setting key 06edee9fe92cf282bfd9249fc4c3347d
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:14:26.296345 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.286586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:48475 (local address 127.1.44.190:36527)
0924 00:14:26.286649 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.286651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.286851 (+ 200us) server_negotiation.cc:407] Connection header received
0924 00:14:26.287461 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.287462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.287474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.287529 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:26.288635 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.289145 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.290681 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.290896 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.290981 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.293528 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:26.293529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:26.293530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:26.293536 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:26.294843 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.295388 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.295389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.295390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.295439 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.295723 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.295723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.295724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.295815 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:26.295861 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.296212 (+ 351us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.296283 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:14:26.296564 5484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.286564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:36527 (local address 127.1.44.129:48475)
0924 00:14:26.286730 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.286771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.287350 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.287613 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.287618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.287944 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:26.288538 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.288544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.289239 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.289240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.290598 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.290600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.290989 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.290990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.291121 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.291690 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:26.291704 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:26.293435 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:26.294928 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:26.294935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:26.294937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:26.295309 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:26.295509 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:26.295509 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:26.295510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:26.295649 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:26.295875 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:26.295881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:26.296043 (+ 162us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.296171 (+ 128us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.296337 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260924 00:14:26.297071 5482 heartbeater.cc:347] Connected to a master server at 127.1.44.190:36527
I20260924 00:14:26.297178 5482 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:26.297472 5482 heartbeater.cc:511] Master 127.1.44.190:36527 requested a full tablet report, sending...
I20260924 00:14:26.298071 5292 ts_manager.cc:194] Registered new tserver with Master: ac1ad4430a7a47dab7b3df79cdaa29a4 (127.1.44.129:37161)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
I20260924 00:14:26.298758 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:48475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:26.313685 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36527
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:26.394047 5487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:26.394258 5487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:26.394282 5487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:26.395560 5487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:26.395606 5487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:26.395635 5487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:26.397145 5487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:26.397396 5487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:26.397636 5487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:26.403142 5495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:26.403084 5492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.403227 5487 server_base.cc:1034] running on GCE node
W20260924 00:14:26.403338 5493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.403565 5487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:26.403838 5487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:26.405012 5487 hybrid_clock.cc:648] HybridClock initialized: now 1790208866405000 us; error 47 us; skew 500 ppm
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:26.407670 5487 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:26.408035 5487 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:26.408093 5487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:26.408186 5487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:26.409206 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "4c1f1e85b96a47dd84b0a8904d50287d"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "97a56ab98261ce4f1094bd4785778857"
server_key_iv: "2eb3527e370b358b4fc8885da1a0f2db"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.409557 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "4c1f1e85b96a47dd84b0a8904d50287d"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "97a56ab98261ce4f1094bd4785778857"
server_key_iv: "2eb3527e370b358b4fc8885da1a0f2db"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.410904 5487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:26.411677 5501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.411868 5487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:26.411939 5487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "4c1f1e85b96a47dd84b0a8904d50287d"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "97a56ab98261ce4f1094bd4785778857"
server_key_iv: "2eb3527e370b358b4fc8885da1a0f2db"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.411990 5487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:26.425419 5487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:26.427791 5487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:26.427909 5487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:26.428100 5487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:26.428437 5487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:26.428474 5487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.428496 5487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:26.428509 5487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.434201 5487 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:34569
I20260924 00:14:26.434233 5614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:34569 every 8 connection(s)
I20260924 00:14:26.434502 5487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:26.438508 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5487
I20260924 00:14:26.438632 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:26.438830 1202 external_mini_cluster.cc:1428] Setting key bd8f4093a84be4653abe976daf5da27d
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:26.444494 5617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.435159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:36527 (local address 127.1.44.130:57689)
0924 00:14:26.435366 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.435403 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.435900 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.436166 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.436170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.436425 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:26.436921 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.436926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.437629 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.437630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.438545 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.438547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.438982 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.438983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.439108 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.439693 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:26.439701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:26.441287 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:26.443121 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:26.443127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:26.443129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:26.443401 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:26.443613 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:26.443614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:26.443614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:26.443710 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:26.444009 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:26.444013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:26.444135 (+ 122us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.444211 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.444330 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":138,"threads_started":1}
I20260924 00:14:26.444507 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.435232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:57689 (local address 127.1.44.190:36527)
0924 00:14:26.435274 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.435276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.435489 (+ 213us) server_negotiation.cc:407] Connection header received
0924 00:14:26.436015 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.436017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.436029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.436084 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:26.437023 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.437523 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.438647 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.438882 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.438977 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.441396 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:26.441397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:26.441397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:26.441403 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:26.443032 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.443470 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.443471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.443472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.443543 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.443783 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.443784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.443784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.443942 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:26.444017 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.444275 (+ 258us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.444366 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:26.444959 5615 heartbeater.cc:347] Connected to a master server at 127.1.44.190:36527
I20260924 00:14:26.445072 5615 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:26.445254 5615 heartbeater.cc:511] Master 127.1.44.190:36527 requested a full tablet report, sending...
I20260924 00:14:26.445603 5292 ts_manager.cc:194] Registered new tserver with Master: 4c1f1e85b96a47dd84b0a8904d50287d (127.1.44.130:34569)
I20260924 00:14:26.445972 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:57689
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:26.465487 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36527
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:35281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:26.545483 5620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:26.545681 5620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:26.545703 5620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:26.546872 5620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:26.546912 5620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:26.546938 5620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:26.548404 5620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:35281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:26.548638 5620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:26.548870 5620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:26.553764 5628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:26.553800 5625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:26.553769 5626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.553828 5620 server_base.cc:1034] running on GCE node
I20260924 00:14:26.554142 5620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:26.554410 5620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:26.555559 5620 hybrid_clock.cc:648] HybridClock initialized: now 1790208866555611 us; error 89 us; skew 500 ppm
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:26.557924 5620 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:26.558290 5620 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:26.558357 5620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:26.558493 5620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:26.559698 5620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "82ad32dd49a1468db50038f852449975"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "4811b580d831e2c76015c6136666232c"
server_key_iv: "15736920e3506d89d791a70ee3452c98"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.560125 5620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "82ad32dd49a1468db50038f852449975"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "4811b580d831e2c76015c6136666232c"
server_key_iv: "15736920e3506d89d791a70ee3452c98"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.561817 5620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:26.562860 5634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.563140 5620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:26.563265 5620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "82ad32dd49a1468db50038f852449975"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "4811b580d831e2c76015c6136666232c"
server_key_iv: "15736920e3506d89d791a70ee3452c98"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.563355 5620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:26.586076 5620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:26.588531 5620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:26.588704 5620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:26.588964 5620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:26.589301 5620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:26.589356 5620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.589388 5620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:26.589418 5620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.594915 5620 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:35147
I20260924 00:14:26.594955 5747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:35147 every 8 connection(s)
I20260924 00:14:26.595243 5620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:26.600476 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5620
I20260924 00:14:26.600585 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:26.600818 1202 external_mini_cluster.cc:1428] Setting key 623b9faaf21bc8ed4a3fec394c4c0906
Sep 24 00:14:26 dist-test-slave-cm7b krb5kdc[5248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208866, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:26.604780 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.595943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:59451 (local address 127.1.44.190:36527)
0924 00:14:26.596031 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.596034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.596212 (+ 178us) server_negotiation.cc:407] Connection header received
0924 00:14:26.596718 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.596719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.596731 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.596783 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:26.597710 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.598204 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.599323 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.599521 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.599612 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.602434 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:26.602435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:26.602436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:26.602442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:26.603663 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.604031 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.604032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.604032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.604086 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.604290 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.604290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.604291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.604386 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:26.604435 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.604661 (+ 226us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.604719 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 00:14:26.605001 5750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.595900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:36527 (local address 127.1.44.131:59451)
0924 00:14:26.596064 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.596106 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.596614 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.596867 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.596870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.597130 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:26.597610 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.597615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.598298 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.598299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.599182 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.599185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.599621 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.599622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.599746 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.600380 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:26.600390 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:26.602359 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:26.603755 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:26.603760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:26.603762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:26.603968 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:26.604142 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:26.604143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:26.604143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:26.604237 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:26.604439 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:26.604442 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:26.604554 (+ 112us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.604633 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.604749 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260924 00:14:26.605439 5748 heartbeater.cc:347] Connected to a master server at 127.1.44.190:36527
I20260924 00:14:26.605540 5748 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:26.605695 5748 heartbeater.cc:511] Master 127.1.44.190:36527 requested a full tablet report, sending...
I20260924 00:14:26.606058 5292 ts_manager.cc:194] Registered new tserver with Master: 82ad32dd49a1468db50038f852449975 (127.1.44.131:35147)
I20260924 00:14:26.606428 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:59451
I20260924 00:14:26.612653 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:14:26.619663 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.613872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.1.44.190:36527)
0924 00:14:26.613954 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.613956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.613972 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:14:26.614036 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.614037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.614046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.614091 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:26.614640 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.615111 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.615918 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.616122 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.616183 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.617320 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:26.617322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:26.617322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:26.617328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:26.618699 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.618973 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.618974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.618975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.619030 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:26.619174 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:26.619174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:26.619174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:26.619357 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:26.619430 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.619517 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.619574 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 00:14:26.621129 5292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:26.622659 5292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:26.628959 5760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.624975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.1.44.131:35147)
0924 00:14:26.625250 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.625253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.625348 (+ 95us) server_negotiation.cc:407] Connection header received
0924 00:14:26.625422 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.625424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.625439 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.625528 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:26.625745 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.626782 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628040 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.628510 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628615 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.628672 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.628821 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.628873 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":89,"threads_started":1}
I20260924 00:14:26.628947 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.624760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:35147 (local address 127.0.0.1:57398)
0924 00:14:26.625296 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.625305 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.625358 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.625615 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.625616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.625619 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:26.625654 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.625656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.626880 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.626881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.627977 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.627979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.628617 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.628697 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.628710 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.628807 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.628873 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":53,"threads_started":1}
I20260924 00:14:26.629007 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.624997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.1.44.130:34569)
0924 00:14:26.625151 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.625155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.625166 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:14:26.625265 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.625268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.625287 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.625375 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:26.625556 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.626681 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.627810 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.628523 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628573 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.628628 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.628870 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.628917 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260924 00:14:26.629101 5759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.624830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.1.44.129:37161)
0924 00:14:26.624972 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.624977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.625253 (+ 276us) server_negotiation.cc:407] Connection header received
0924 00:14:26.625518 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.625521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.625535 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.625609 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:26.625818 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.626549 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.627959 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.628654 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628728 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.628780 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.628969 (+ 189us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.629010 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260924 00:14:26.629209 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.624734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:34569 (local address 127.0.0.1:60512)
0924 00:14:26.625000 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.625051 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.625136 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.625392 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.625395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.625398 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:26.625466 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.625468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.626792 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.626793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.627718 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.627721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.628693 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.628764 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.628768 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.629090 (+ 322us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.629138 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":5,"thread_start_us":75,"threads_started":1}
I20260924 00:14:26.629289 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.624735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37161 (local address 127.0.0.1:48360)
0924 00:14:26.625154 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.625164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.625422 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.625626 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.625627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.625629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:26.625679 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.625682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.626646 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.626647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.627871 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.627873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.628811 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.628812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.628882 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.628885 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.629170 (+ 285us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.629203 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":77,"threads_started":1}
I20260924 00:14:26.630245 5549 tablet_service.cc:1467] Processing CreateTablet for tablet c9fc0aab4e0d49339b612155c5d18961 (DEFAULT_TABLE table=test-table [id=264bcd8dc746474a8a7f9648d708ee88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:26.630256 5416 tablet_service.cc:1467] Processing CreateTablet for tablet c9fc0aab4e0d49339b612155c5d18961 (DEFAULT_TABLE table=test-table [id=264bcd8dc746474a8a7f9648d708ee88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:26.630231 5682 tablet_service.cc:1467] Processing CreateTablet for tablet c9fc0aab4e0d49339b612155c5d18961 (DEFAULT_TABLE table=test-table [id=264bcd8dc746474a8a7f9648d708ee88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:26.630589 5682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fc0aab4e0d49339b612155c5d18961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:26.630596 5549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fc0aab4e0d49339b612155c5d18961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:26.630597 5416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fc0aab4e0d49339b612155c5d18961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:26.633337 5762 tablet_bootstrap.cc:492] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Bootstrap starting.
I20260924 00:14:26.633800 5764 tablet_bootstrap.cc:492] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4: Bootstrap starting.
I20260924 00:14:26.634131 5762 tablet_bootstrap.cc:654] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:26.634430 5764 tablet_bootstrap.cc:654] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:26.634785 5762 log.cc:824] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:26.635051 5764 log.cc:824] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:26.635424 5763 tablet_bootstrap.cc:492] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Bootstrap starting.
I20260924 00:14:26.635663 5762 tablet_bootstrap.cc:492] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: No bootstrap required, opened a new log
I20260924 00:14:26.635742 5762 ts_tablet_manager.cc:1397] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:26.636003 5763 tablet_bootstrap.cc:654] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:26.636616 5763 log.cc:824] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:26.637151 5764 tablet_bootstrap.cc:492] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4: No bootstrap required, opened a new log
I20260924 00:14:26.637204 5764 ts_tablet_manager.cc:1397] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:14:26.637512 5762 raft_consensus.cc:348] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.637669 5762 raft_consensus.cc:374] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:26.637715 5762 raft_consensus.cc:729] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1f1e85b96a47dd84b0a8904d50287d, State: Initialized, Role: FOLLOWER
I20260924 00:14:26.637835 5762 consensus_queue.cc:260] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.638128 5762 ts_tablet_manager.cc:1428] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:14:26.638136 5615 heartbeater.cc:503] Master 127.1.44.190:36527 was elected leader, sending a full tablet report...
I20260924 00:14:26.639065 5764 raft_consensus.cc:348] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.639283 5764 raft_consensus.cc:374] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:26.639333 5764 raft_consensus.cc:729] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1ad4430a7a47dab7b3df79cdaa29a4, State: Initialized, Role: FOLLOWER
I20260924 00:14:26.639443 5764 consensus_queue.cc:260] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.639591 5763 tablet_bootstrap.cc:492] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: No bootstrap required, opened a new log
I20260924 00:14:26.639663 5763 ts_tablet_manager.cc:1397] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 00:14:26.639729 5764 ts_tablet_manager.cc:1428] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:26.639756 5482 heartbeater.cc:503] Master 127.1.44.190:36527 was elected leader, sending a full tablet report...
I20260924 00:14:26.641403 5763 raft_consensus.cc:348] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.641561 5763 raft_consensus.cc:374] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:26.641607 5763 raft_consensus.cc:729] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ad32dd49a1468db50038f852449975, State: Initialized, Role: FOLLOWER
I20260924 00:14:26.641719 5763 consensus_queue.cc:260] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.641968 5763 ts_tablet_manager.cc:1428] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:14:26.642057 5748 heartbeater.cc:503] Master 127.1.44.190:36527 was elected leader, sending a full tablet report...
W20260924 00:14:26.684994 5616 tablet.cc:2369] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:26.741441 5770 raft_consensus.cc:482] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:26.741609 5770 raft_consensus.cc:504] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.741978 5770 leader_election.cc:288] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c1f1e85b96a47dd84b0a8904d50287d (127.1.44.130:34569), ac1ad4430a7a47dab7b3df79cdaa29a4 (127.1.44.129:37161)
I20260924 00:14:26.744867 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.742303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:35183 (local address 127.1.44.130:34569)
0924 00:14:26.742346 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.742348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.742356 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:26.742409 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.742410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.742421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.742494 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:26.742667 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.743155 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.744142 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.744512 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.744598 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.744608 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.744768 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.744803 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:26.744868 5750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.742197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:34569 (local address 127.1.44.131:35183)
0924 00:14:26.742276 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.742291 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.742352 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.742510 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.742511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.742512 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:26.742580 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.742582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.743289 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.743290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.744041 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.744043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.744595 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.744596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.744683 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.744686 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.744760 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.744798 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260924 00:14:26.744961 5759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.742286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:42895 (local address 127.1.44.129:37161)
0924 00:14:26.742336 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.742338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.742480 (+ 142us) server_negotiation.cc:407] Connection header received
0924 00:14:26.742558 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.742559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.742571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.742611 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:26.742771 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.743297 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.744215 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.744598 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.744682 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.744691 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.744837 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.744871 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 00:14:26.744979 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.742197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37161 (local address 127.1.44.131:42895)
0924 00:14:26.742401 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.742416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:26.742489 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:26.742622 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:26.742623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:26.742624 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:26.742692 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.742694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.743376 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.743377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.744124 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:26.744126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.744682 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.744682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:26.744758 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.744766 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:14:26.744834 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 00:14:26.744871 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260924 00:14:26.745223 5569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fc0aab4e0d49339b612155c5d18961" candidate_uuid: "82ad32dd49a1468db50038f852449975" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" is_pre_election: true
I20260924 00:14:26.745345 5569 raft_consensus.cc:2454] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82ad32dd49a1468db50038f852449975 in term 0.
I20260924 00:14:26.745615 5637 leader_election.cc:302] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1f1e85b96a47dd84b0a8904d50287d, 82ad32dd49a1468db50038f852449975; no voters:
I20260924 00:14:26.745635 5436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fc0aab4e0d49339b612155c5d18961" candidate_uuid: "82ad32dd49a1468db50038f852449975" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" is_pre_election: true
I20260924 00:14:26.745779 5436 raft_consensus.cc:2454] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82ad32dd49a1468db50038f852449975 in term 0.
I20260924 00:14:26.745775 5770 raft_consensus.cc:2781] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:26.745848 5770 raft_consensus.cc:482] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:26.745872 5770 raft_consensus.cc:3037] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:26.746592 5770 raft_consensus.cc:504] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.746723 5770 leader_election.cc:288] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [CANDIDATE]: Term 1 election: Requested vote from peers 4c1f1e85b96a47dd84b0a8904d50287d (127.1.44.130:34569), ac1ad4430a7a47dab7b3df79cdaa29a4 (127.1.44.129:37161)
I20260924 00:14:26.746862 5569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fc0aab4e0d49339b612155c5d18961" candidate_uuid: "82ad32dd49a1468db50038f852449975" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1f1e85b96a47dd84b0a8904d50287d"
I20260924 00:14:26.746977 5569 raft_consensus.cc:3037] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:26.747694 5569 raft_consensus.cc:2454] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82ad32dd49a1468db50038f852449975 in term 1.
I20260924 00:14:26.747814 5436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fc0aab4e0d49339b612155c5d18961" candidate_uuid: "82ad32dd49a1468db50038f852449975" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4"
I20260924 00:14:26.747889 5637 leader_election.cc:302] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1f1e85b96a47dd84b0a8904d50287d, 82ad32dd49a1468db50038f852449975; no voters:
I20260924 00:14:26.747905 5436 raft_consensus.cc:3037] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:26.747988 5770 raft_consensus.cc:2781] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:26.748174 5770 raft_consensus.cc:686] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 1 LEADER]: Becoming Leader. State: Replica: 82ad32dd49a1468db50038f852449975, State: Running, Role: LEADER
I20260924 00:14:26.748278 5770 consensus_queue.cc:237] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } }
I20260924 00:14:26.748791 5436 raft_consensus.cc:2454] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82ad32dd49a1468db50038f852449975 in term 1.
I20260924 00:14:26.749291 5291 catalog_manager.cc:5495] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82ad32dd49a1468db50038f852449975 (127.1.44.131). New cstate: current_term: 1 leader_uuid: "82ad32dd49a1468db50038f852449975" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82ad32dd49a1468db50038f852449975" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 35147 } health_report { overall_health: HEALTHY } } }
I20260924 00:14:26.772977 5760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.770547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.1.44.131:35147)
0924 00:14:26.770600 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.770602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.770612 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:26.770689 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.770690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.770710 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.770802 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:26.770934 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.771431 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.772191 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:26.772396 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:26.772470 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:26.772639 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:26.772746 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:26.772837 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.772861 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.772879 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 00:14:26.774770 5569 raft_consensus.cc:1261] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 1 FOLLOWER]: Refusing update from remote peer 82ad32dd49a1468db50038f852449975: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:26.774773 5436 raft_consensus.cc:1261] T c9fc0aab4e0d49339b612155c5d18961 P ac1ad4430a7a47dab7b3df79cdaa29a4 [term 1 FOLLOWER]: Refusing update from remote peer 82ad32dd49a1468db50038f852449975: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:26.775121 5772 consensus_queue.cc:1035] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1f1e85b96a47dd84b0a8904d50287d" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:26.775293 5770 consensus_queue.cc:1035] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac1ad4430a7a47dab7b3df79cdaa29a4" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:26.778153 5776 mvcc.cc:205] Tried to move back new op lower bound from 7332695518306914304 to 7332695518201937920. Current Snapshot: MvccSnapshot[applied={T|T < 7332695518306914304 or (T in {7332695518306914304})}]
I20260924 00:14:26.779771 5291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:26.779877 5291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:26.780625 5291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9640d19265145c198d253461480ba61: Sending DeleteTablet for 3 replicas of tablet c9fc0aab4e0d49339b612155c5d18961
I20260924 00:14:26.780942 5682 tablet_service.cc:1514] Processing DeleteTablet for tablet c9fc0aab4e0d49339b612155c5d18961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:26 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:57398
I20260924 00:14:26.780962 5416 tablet_service.cc:1514] Processing DeleteTablet for tablet c9fc0aab4e0d49339b612155c5d18961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:26 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:48360
I20260924 00:14:26.781131 5785 tablet_replica.cc:330] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: stopping tablet replica
I20260924 00:14:26.781217 5785 raft_consensus.cc:2229] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:26.781226 5549 tablet_service.cc:1514] Processing DeleteTablet for tablet c9fc0aab4e0d49339b612155c5d18961 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:26 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:60512
I20260924 00:14:26.781296 5785 raft_consensus.cc:2258] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:26.781356 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5354
I20260924 00:14:26.783568 5785 ts_tablet_manager.cc:1905] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:26.785262 5785 ts_tablet_manager.cc:1918] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:26.785336 5785 log.cc:1196] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/c9fc0aab4e0d49339b612155c5d18961
I20260924 00:14:26.785596 5785 ts_tablet_manager.cc:1939] T c9fc0aab4e0d49339b612155c5d18961 P 82ad32dd49a1468db50038f852449975: Deleting consensus metadata
I20260924 00:14:26.786005 5276 catalog_manager.cc:4841] TS 82ad32dd49a1468db50038f852449975 (127.1.44.131:35147): tablet c9fc0aab4e0d49339b612155c5d18961 (table test-table [id=264bcd8dc746474a8a7f9648d708ee88]) successfully deleted
I20260924 00:14:26.786036 5787 tablet_replica.cc:330] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: stopping tablet replica
I20260924 00:14:26.786134 5787 raft_consensus.cc:2229] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:26.786239 5787 raft_consensus.cc:2258] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:26.786612 5787 ts_tablet_manager.cc:1905] T c9fc0aab4e0d49339b612155c5d18961 P 4c1f1e85b96a47dd84b0a8904d50287d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:14:26.786752 5279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:37161 (error 108)
I20260924 00:14:26.787413 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5487
I20260924 00:14:26.787434 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.787309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37161 (local address 127.0.0.1:48362)
0924 00:14:26.787352 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.787400 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 00:14:26.791782 5279 catalog_manager.cc:4584] TS ac1ad4430a7a47dab7b3df79cdaa29a4 (127.1.44.129:37161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9fc0aab4e0d49339b612155c5d18961: Network error: Client connection negotiation failed: client connection to 127.1.44.129:37161: connect: Connection refused (error 111)
I20260924 00:14:26.792879 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.792769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:34569 (local address 127.0.0.1:60516)
0924 00:14:26.792821 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:26.792847 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.130:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 00:14:26.792886 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5620
I20260924 00:14:26.797330 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5261
2026-09-24T00:14:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1972 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T00:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:26Z Disabled control of system clock
I20260924 00:14:26.824381 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:45705
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 00:14:26.902369 5792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:26.902576 5792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:26.902597 5792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:26.903752 5792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:26.903791 5792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:26.903805 5792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:26.903817 5792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:26.903829 5792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:26.905081 5792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:45705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:26.905261 5792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:26.905475 5792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:26.910230 5798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.910347 5792 server_base.cc:1034] running on GCE node
W20260924 00:14:26.910248 5800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:26.910254 5797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:26.910735 5792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:26.911047 5792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:26.912220 5792 hybrid_clock.cc:648] HybridClock initialized: now 1790208866912213 us; error 46 us; skew 500 ppm
I20260924 00:14:26.913437 5792 webserver.cc:460] Webserver started at http://127.1.44.190:40227/ using document root <none> and password file <none>
I20260924 00:14:26.913687 5792 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:26.913763 5792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:26.913904 5792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:26.914970 5792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "806344528faa4c5dacb02fee6fa46f06"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "61b2bd196f9e728b8bcadac13f681f55"
server_key_iv: "003e50f17379617abbaecb40663c31b9"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.915362 5792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "806344528faa4c5dacb02fee6fa46f06"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "61b2bd196f9e728b8bcadac13f681f55"
server_key_iv: "003e50f17379617abbaecb40663c31b9"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.916688 5792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:26.917397 5806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:26.917554 5792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:26.917652 5792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "806344528faa4c5dacb02fee6fa46f06"
format_stamp: "Formatted at 2026-09-24 00:14:26 on dist-test-slave-cm7b"
server_key: "61b2bd196f9e728b8bcadac13f681f55"
server_key_iv: "003e50f17379617abbaecb40663c31b9"
server_key_version: "encryptionkey@0"
I20260924 00:14:26.917737 5792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:26.921118 5792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:26.921373 5792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:26.921518 5792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:26.925184 5792 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:45705
I20260924 00:14:26.925212 5858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:45705 every 8 connection(s)
I20260924 00:14:26.925534 5792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:26.926169 5859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:26.928157 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5792
I20260924 00:14:26.928233 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:26.928409 1202 external_mini_cluster.cc:1428] Setting key 4b98973345b458a1a1e0f0eb1542357f
I20260924 00:14:26.929114 5859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06: Bootstrap starting.
I20260924 00:14:26.929699 5859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:26.930027 5859 log.cc:824] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:26.930819 5859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06: No bootstrap required, opened a new log
I20260924 00:14:26.930920 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:26.928923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.1.44.190:45705)
0924 00:14:26.929114 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:26.929121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:26.929143 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:14:26.929558 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:26.929564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:26.929602 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:26.929714 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:26.930453 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:26.930456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:26.930456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:26.930461 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:26.930527 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:26.930542 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:26.930614 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:26.930678 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 00:14:26.930731 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260924 00:14:26.932323 5859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806344528faa4c5dacb02fee6fa46f06" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45705 } }
I20260924 00:14:26.932446 5859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:26.932467 5859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 806344528faa4c5dacb02fee6fa46f06, State: Initialized, Role: FOLLOWER
I20260924 00:14:26.932551 5859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806344528faa4c5dacb02fee6fa46f06" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45705 } }
I20260924 00:14:26.932863 5859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:26.932885 5859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:26.932914 5859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:26.933363 5859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806344528faa4c5dacb02fee6fa46f06" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45705 } }
I20260924 00:14:26.933441 5859 leader_election.cc:302] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 806344528faa4c5dacb02fee6fa46f06; no voters:
I20260924 00:14:26.933557 5859 leader_election.cc:288] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:26.933606 5864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:26.933779 5864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [term 1 LEADER]: Becoming Leader. State: Replica: 806344528faa4c5dacb02fee6fa46f06, State: Running, Role: LEADER
I20260924 00:14:26.933871 5859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:26.933873 5864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806344528faa4c5dacb02fee6fa46f06" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45705 } }
I20260924 00:14:26.934302 5865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "806344528faa4c5dacb02fee6fa46f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806344528faa4c5dacb02fee6fa46f06" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45705 } } }
I20260924 00:14:26.934300 5866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 806344528faa4c5dacb02fee6fa46f06. Latest consensus state: current_term: 1 leader_uuid: "806344528faa4c5dacb02fee6fa46f06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806344528faa4c5dacb02fee6fa46f06" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45705 } } }
I20260924 00:14:26.934522 5865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:26.934540 5866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:26.934952 5874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:26.935431 5874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:26.936820 5874 catalog_manager.cc:1347] Generated new cluster ID: b9d8d4b193764e2cb1dc54eedfed7cc4
I20260924 00:14:26.936872 5874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:26.943499 5874 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:26.943655 5874 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:26.948685 5874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06: Generated new TSK 0
I20260924 00:14:26.948882 5874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:26.951685 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:45705
--builtin_ntp_servers=127.1.44.148:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 00:14:27.030929 5883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:27.031127 5883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:27.031149 5883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:27.032325 5883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:27.032371 5883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:27.032397 5883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:27.033767 5883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:27.034013 5883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:27.034341 5883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:27.039352 5888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.039546 5891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.039343 5889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.039834 5883 server_base.cc:1034] running on GCE node
I20260924 00:14:27.040056 5883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:27.040330 5883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:27.041525 5883 hybrid_clock.cc:648] HybridClock initialized: now 1790208867041510 us; error 44 us; skew 500 ppm
I20260924 00:14:27.042922 5883 webserver.cc:460] Webserver started at http://127.1.44.129:38779/ using document root <none> and password file <none>
I20260924 00:14:27.043223 5883 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:27.043291 5883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:27.043426 5883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:27.044786 5883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "36de5b678cb44030a2d7b3121ff81c58"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "88845393efa791126d1fd0f660804285"
server_key_iv: "bd34416a61c7c31841374bb44880266c"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.045208 5883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "36de5b678cb44030a2d7b3121ff81c58"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "88845393efa791126d1fd0f660804285"
server_key_iv: "bd34416a61c7c31841374bb44880266c"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.046773 5883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:27.047610 5897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.047791 5883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.047875 5883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "36de5b678cb44030a2d7b3121ff81c58"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "88845393efa791126d1fd0f660804285"
server_key_iv: "bd34416a61c7c31841374bb44880266c"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.047947 5883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:27.052433 5883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:27.052699 5883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:27.052821 5883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:27.053025 5883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:27.053320 5883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:27.053359 5883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.053386 5883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:27.053403 5883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.060550 5883 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:37833
I20260924 00:14:27.060631 6010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:37833 every 8 connection(s)
I20260924 00:14:27.060952 5883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:27.064184 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.061822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:57121 (local address 127.1.44.190:45705)
0924 00:14:27.061863 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.061865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.062100 (+ 235us) server_negotiation.cc:407] Connection header received
0924 00:14:27.062612 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.062614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.062620 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.062672 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.063883 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.063885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.063885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.063899 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.063921 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.063926 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.064007 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.064099 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.064114 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:27.064381 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.061807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:45705 (local address 127.1.44.129:57121)
0924 00:14:27.061979 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.062010 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.062497 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.062740 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.062750 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.062758 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.063719 (+ 961us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.063719 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.063747 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.063775 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.064007 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.064013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.064016 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.064110 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.064170 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260924 00:14:27.064854 6011 heartbeater.cc:347] Connected to a master server at 127.1.44.190:45705
I20260924 00:14:27.064942 6011 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:27.065054 6011 heartbeater.cc:511] Master 127.1.44.190:45705 requested a full tablet report, sending...
I20260924 00:14:27.065513 5823 ts_manager.cc:194] Registered new tserver with Master: 36de5b678cb44030a2d7b3121ff81c58 (127.1.44.129:37833)
I20260924 00:14:27.066293 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 5883
I20260924 00:14:27.066377 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:27.066567 1202 external_mini_cluster.cc:1428] Setting key a2ae79b9c58dbb384735fadc4aaa68af
I20260924 00:14:27.067497 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:45705
--builtin_ntp_servers=127.1.44.148:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 00:14:27.147862 6014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:27.148068 6014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:27.148090 6014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:27.149181 6014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:27.149216 6014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:27.149242 6014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:27.150485 6014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:27.150665 6014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:27.150890 6014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:27.155930 6022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.156018 6014 server_base.cc:1034] running on GCE node
W20260924 00:14:27.155982 6020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.155974 6019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.156371 6014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:27.156617 6014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:27.157787 6014 hybrid_clock.cc:648] HybridClock initialized: now 1790208867157777 us; error 28 us; skew 500 ppm
I20260924 00:14:27.159166 6014 webserver.cc:460] Webserver started at http://127.1.44.130:40061/ using document root <none> and password file <none>
I20260924 00:14:27.159466 6014 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:27.159524 6014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:27.159651 6014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:27.160689 6014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "ab031ddb50384fdb83abe63763b7d98a"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "d34eba4c49c7075e3796cfbecd2ad2da"
server_key_iv: "9619d9c007a46cdcf26df6e71e538ea1"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.161041 6014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "ab031ddb50384fdb83abe63763b7d98a"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "d34eba4c49c7075e3796cfbecd2ad2da"
server_key_iv: "9619d9c007a46cdcf26df6e71e538ea1"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.162348 6014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:27.163067 6028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.163262 6014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:27.163345 6014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "ab031ddb50384fdb83abe63763b7d98a"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "d34eba4c49c7075e3796cfbecd2ad2da"
server_key_iv: "9619d9c007a46cdcf26df6e71e538ea1"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.163406 6014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:27.166577 6014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:27.166775 6014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:27.166864 6014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:27.167002 6014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:27.167294 6014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:27.167327 6014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.167347 6014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:27.167359 6014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.172931 6014 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:43725
I20260924 00:14:27.173003 6141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:43725 every 8 connection(s)
I20260924 00:14:27.173295 6014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:27.176107 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.174008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:52559 (local address 127.1.44.190:45705)
0924 00:14:27.174074 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.174080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.174291 (+ 211us) server_negotiation.cc:407] Connection header received
0924 00:14:27.174735 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.174736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.174741 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.174791 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.175907 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.175908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.175909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.175913 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.175926 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.175930 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.175980 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.176045 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.176056 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 00:14:27.176261 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.174008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:45705 (local address 127.1.44.130:52559)
0924 00:14:27.174191 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.174222 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.174650 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.174840 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.174843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.174850 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.175774 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.175774 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.175799 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.175824 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.175981 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.175983 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.175985 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.176037 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.176080 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260924 00:14:27.176712 6142 heartbeater.cc:347] Connected to a master server at 127.1.44.190:45705
I20260924 00:14:27.176815 6142 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:27.176955 6142 heartbeater.cc:511] Master 127.1.44.190:45705 requested a full tablet report, sending...
I20260924 00:14:27.177253 5823 ts_manager.cc:194] Registered new tserver with Master: ab031ddb50384fdb83abe63763b7d98a (127.1.44.130:43725)
I20260924 00:14:27.181746 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6014
I20260924 00:14:27.181831 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:27.182032 1202 external_mini_cluster.cc:1428] Setting key f964906663ed2d741dbce594e700f8f0
I20260924 00:14:27.182956 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:45705
--builtin_ntp_servers=127.1.44.148:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 00:14:27.261744 6145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:27.261932 6145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:27.261953 6145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:27.263091 6145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:27.263129 6145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:27.263154 6145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:27.264468 6145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:27.264704 6145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:27.264978 6145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:27.269925 6150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.269903 6151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.269996 6145 server_base.cc:1034] running on GCE node
W20260924 00:14:27.269899 6153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.270257 6145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:27.270491 6145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:27.271646 6145 hybrid_clock.cc:648] HybridClock initialized: now 1790208867271633 us; error 44 us; skew 500 ppm
I20260924 00:14:27.272806 6145 webserver.cc:460] Webserver started at http://127.1.44.131:39091/ using document root <none> and password file <none>
I20260924 00:14:27.273046 6145 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:27.273114 6145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:27.273269 6145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:27.274274 6145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "1190e14133c241bb941a1cc755e9f3e2"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "d1f21e47ce192af26bec4f5f667227f5"
server_key_iv: "c77ac26c12b18ffc8f0d1e8abbb02168"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.274624 6145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "1190e14133c241bb941a1cc755e9f3e2"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "d1f21e47ce192af26bec4f5f667227f5"
server_key_iv: "c77ac26c12b18ffc8f0d1e8abbb02168"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.275955 6145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:27.276654 6159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.276814 6145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:27.276911 6145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "1190e14133c241bb941a1cc755e9f3e2"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "d1f21e47ce192af26bec4f5f667227f5"
server_key_iv: "c77ac26c12b18ffc8f0d1e8abbb02168"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.277000 6145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:27.280254 6145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:27.280493 6145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:27.280634 6145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:27.280835 6145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:27.281124 6145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:27.281178 6145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.281216 6145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:27.281243 6145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.286728 6145 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:39229
I20260924 00:14:27.286792 6272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:39229 every 8 connection(s)
I20260924 00:14:27.287103 6145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:27.289994 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.287836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:39915 (local address 127.1.44.190:45705)
0924 00:14:27.287929 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.287931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.288114 (+ 183us) server_negotiation.cc:407] Connection header received
0924 00:14:27.288597 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.288599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.288605 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.288656 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.289741 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.289742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.289743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.289747 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.289761 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.289765 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.289817 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.289903 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.289916 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 00:14:27.290169 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.287797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:45705 (local address 127.1.44.131:39915)
0924 00:14:27.287986 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.288031 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.288487 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.288719 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.288723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.288732 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.289591 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.289591 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.289617 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.289642 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.289826 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.289831 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.289833 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.289911 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.289968 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260924 00:14:27.290645 6273 heartbeater.cc:347] Connected to a master server at 127.1.44.190:45705
I20260924 00:14:27.290732 6273 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:27.290886 6273 heartbeater.cc:511] Master 127.1.44.190:45705 requested a full tablet report, sending...
I20260924 00:14:27.291285 5823 ts_manager.cc:194] Registered new tserver with Master: 1190e14133c241bb941a1cc755e9f3e2 (127.1.44.131:39229)
I20260924 00:14:27.297307 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6145
I20260924 00:14:27.297391 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:27.297528 1202 external_mini_cluster.cc:1428] Setting key fbd8346de43300d841c665754c580ddf
I20260924 00:14:27.298292 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:14:27.300585 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.299545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.1.44.190:45705)
0924 00:14:27.299605 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.299606 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.299703 (+ 97us) server_negotiation.cc:407] Connection header received
0924 00:14:27.299756 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.299757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.299761 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.299814 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.300305 (+ 491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.300307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.300307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.300311 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.300327 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.300332 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.300385 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.300470 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.300483 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:14:27.302101 5823 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:27.303666 5823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:27.307034 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.305644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37833 (local address 127.0.0.1:32970)
0924 00:14:27.305793 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.305804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.305850 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.306111 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.306115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.306122 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.306577 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.306577 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.306592 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.306613 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.306838 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.306845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.306848 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.306918 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.306932 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260924 00:14:27.307269 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.305802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.1.44.129:37833)
0924 00:14:27.305975 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.305979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.305987 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:27.306008 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.306010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.306017 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.306095 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.306697 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.306703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.306704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.306708 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.306744 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.306769 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.306845 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.307154 (+ 309us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.307174 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260924 00:14:27.307890 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.305609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43725 (local address 127.0.0.1:59608)
0924 00:14:27.305737 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.305777 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.305831 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.306872 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.306873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.306876 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.307483 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.307483 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.307488 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.307498 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.307748 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.307749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.307750 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.307801 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.307815 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":82,"threads_started":1}
I20260924 00:14:27.307898 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.305773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.1.44.130:43725)
0924 00:14:27.306523 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.306527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.306536 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:27.306559 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.306561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.306570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.306639 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.307609 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.307615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.307616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.307620 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.307648 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.307678 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.307742 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.307806 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.307820 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":79,"threads_started":1}
I20260924 00:14:27.307953 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.306120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.1.44.131:39229)
0924 00:14:27.306369 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.306374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.306384 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:27.306408 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.306410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.306418 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.306493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.307534 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.307539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.307540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.307543 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.307598 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.307623 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.307677 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.307848 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.307862 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":71,"threads_started":1}
I20260924 00:14:27.307936 5945 tablet_service.cc:1467] Processing CreateTablet for tablet 1afe7df0af5b45d2a3ab32e50bcf2542 (DEFAULT_TABLE table=test-table [id=bbbe2e0a0a044d8894fef939fc395e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:27.308327 5945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1afe7df0af5b45d2a3ab32e50bcf2542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:27.308292 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.306048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:39229 (local address 127.0.0.1:53480)
0924 00:14:27.306169 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.306179 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.306218 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.306497 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.306497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.306500 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.307252 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.307253 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.307260 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.307271 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.307711 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.307712 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.307713 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.307759 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.307771 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260924 00:14:27.309055 6076 tablet_service.cc:1467] Processing CreateTablet for tablet 1afe7df0af5b45d2a3ab32e50bcf2542 (DEFAULT_TABLE table=test-table [id=bbbe2e0a0a044d8894fef939fc395e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:27.309049 6207 tablet_service.cc:1467] Processing CreateTablet for tablet 1afe7df0af5b45d2a3ab32e50bcf2542 (DEFAULT_TABLE table=test-table [id=bbbe2e0a0a044d8894fef939fc395e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:27.309358 6076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1afe7df0af5b45d2a3ab32e50bcf2542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:27.309358 6207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1afe7df0af5b45d2a3ab32e50bcf2542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:27.310699 6287 tablet_bootstrap.cc:492] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: Bootstrap starting.
I20260924 00:14:27.311448 6287 tablet_bootstrap.cc:654] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:27.311756 6287 log.cc:824] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:27.312423 6289 tablet_bootstrap.cc:492] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Bootstrap starting.
I20260924 00:14:27.312529 6287 tablet_bootstrap.cc:492] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: No bootstrap required, opened a new log
I20260924 00:14:27.312597 6287 ts_tablet_manager.cc:1397] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:27.312765 6290 tablet_bootstrap.cc:492] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Bootstrap starting.
I20260924 00:14:27.313097 6289 tablet_bootstrap.cc:654] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:27.313351 6290 tablet_bootstrap.cc:654] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:27.313541 6289 log.cc:824] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:27.313689 6290 log.cc:824] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:27.314309 6289 tablet_bootstrap.cc:492] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: No bootstrap required, opened a new log
I20260924 00:14:27.314383 6289 ts_tablet_manager.cc:1397] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:27.315335 6287 raft_consensus.cc:348] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.315508 6287 raft_consensus.cc:374] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:27.315544 6287 raft_consensus.cc:729] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36de5b678cb44030a2d7b3121ff81c58, State: Initialized, Role: FOLLOWER
I20260924 00:14:27.315680 6287 consensus_queue.cc:260] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.315732 6290 tablet_bootstrap.cc:492] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: No bootstrap required, opened a new log
I20260924 00:14:27.315802 6290 ts_tablet_manager.cc:1397] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 00:14:27.315928 6287 ts_tablet_manager.cc:1428] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260924 00:14:27.316045 6011 heartbeater.cc:503] Master 127.1.44.190:45705 was elected leader, sending a full tablet report...
I20260924 00:14:27.317028 6289 raft_consensus.cc:348] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.317174 6289 raft_consensus.cc:374] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:27.317224 6289 raft_consensus.cc:729] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1190e14133c241bb941a1cc755e9f3e2, State: Initialized, Role: FOLLOWER
I20260924 00:14:27.317340 6289 consensus_queue.cc:260] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.317592 6289 ts_tablet_manager.cc:1428] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260924 00:14:27.317621 6273 heartbeater.cc:503] Master 127.1.44.190:45705 was elected leader, sending a full tablet report...
I20260924 00:14:27.318657 6290 raft_consensus.cc:348] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.318769 6290 raft_consensus.cc:374] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:27.318789 6290 raft_consensus.cc:729] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab031ddb50384fdb83abe63763b7d98a, State: Initialized, Role: FOLLOWER
I20260924 00:14:27.318909 6290 consensus_queue.cc:260] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.319177 6290 ts_tablet_manager.cc:1428] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:27.319190 6142 heartbeater.cc:503] Master 127.1.44.190:45705 was elected leader, sending a full tablet report...
W20260924 00:14:27.423827 6143 tablet.cc:2369] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:27.450580 6295 raft_consensus.cc:482] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:27.450698 6295 raft_consensus.cc:504] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.451143 6295 leader_election.cc:288] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36de5b678cb44030a2d7b3121ff81c58 (127.1.44.129:37833), 1190e14133c241bb941a1cc755e9f3e2 (127.1.44.131:39229)
I20260924 00:14:27.452581 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.451485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:39341 (local address 127.1.44.129:37833)
0924 00:14:27.451541 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.451544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.451551 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:27.451578 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.451579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.451584 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.451639 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.452297 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.452299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.452300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.452304 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.452323 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.452328 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.452418 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.452481 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.452495 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:14:27.452597 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.451352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37833 (local address 127.1.44.130:39341)
0924 00:14:27.451397 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.451408 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.451452 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.451657 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.451658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.451660 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.452193 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.452193 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.452201 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.452214 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.452417 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.452418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.452419 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.452477 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.452491 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 00:14:27.452669 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.451489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:53811 (local address 127.1.44.131:39229)
0924 00:14:27.451538 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.451540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.451655 (+ 115us) server_negotiation.cc:407] Connection header received
0924 00:14:27.451723 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.451724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.451728 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.451803 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.452426 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.452427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.452427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.452432 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.452446 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.452450 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.452483 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.452588 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.452600 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:27.452838 6296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.451376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:39229 (local address 127.1.44.130:53811)
0924 00:14:27.451565 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.451577 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.451656 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.451805 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.451806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.451808 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:27.452325 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:27.452325 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:27.452330 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:27.452339 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:27.452525 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:27.452526 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:27.452526 (+ 0us) client_negotiation.cc:769] Sending connection context
0924 00:14:27.452754 (+ 228us) client_negotiation.cc:240] Negotiation successful
0924 00:14:27.452767 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":87,"threads_started":1}
I20260924 00:14:27.452930 5965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afe7df0af5b45d2a3ab32e50bcf2542" candidate_uuid: "ab031ddb50384fdb83abe63763b7d98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36de5b678cb44030a2d7b3121ff81c58" is_pre_election: true
I20260924 00:14:27.453087 5965 raft_consensus.cc:2454] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab031ddb50384fdb83abe63763b7d98a in term 0.
I20260924 00:14:27.453107 6227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afe7df0af5b45d2a3ab32e50bcf2542" candidate_uuid: "ab031ddb50384fdb83abe63763b7d98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1190e14133c241bb941a1cc755e9f3e2" is_pre_election: true
I20260924 00:14:27.453251 6227 raft_consensus.cc:2454] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab031ddb50384fdb83abe63763b7d98a in term 0.
I20260924 00:14:27.453331 6029 leader_election.cc:302] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36de5b678cb44030a2d7b3121ff81c58, ab031ddb50384fdb83abe63763b7d98a; no voters:
I20260924 00:14:27.453459 6295 raft_consensus.cc:2781] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:27.453513 6295 raft_consensus.cc:482] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:27.453534 6295 raft_consensus.cc:3037] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:27.454210 6295 raft_consensus.cc:504] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.454342 6295 leader_election.cc:288] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [CANDIDATE]: Term 1 election: Requested vote from peers 36de5b678cb44030a2d7b3121ff81c58 (127.1.44.129:37833), 1190e14133c241bb941a1cc755e9f3e2 (127.1.44.131:39229)
I20260924 00:14:27.454469 5965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afe7df0af5b45d2a3ab32e50bcf2542" candidate_uuid: "ab031ddb50384fdb83abe63763b7d98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36de5b678cb44030a2d7b3121ff81c58"
I20260924 00:14:27.454489 6227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afe7df0af5b45d2a3ab32e50bcf2542" candidate_uuid: "ab031ddb50384fdb83abe63763b7d98a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1190e14133c241bb941a1cc755e9f3e2"
I20260924 00:14:27.454538 5965 raft_consensus.cc:3037] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:27.454545 6227 raft_consensus.cc:3037] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:27.455421 5965 raft_consensus.cc:2454] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab031ddb50384fdb83abe63763b7d98a in term 1.
I20260924 00:14:27.455421 6227 raft_consensus.cc:2454] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab031ddb50384fdb83abe63763b7d98a in term 1.
I20260924 00:14:27.455627 6029 leader_election.cc:302] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36de5b678cb44030a2d7b3121ff81c58, ab031ddb50384fdb83abe63763b7d98a; no voters:
I20260924 00:14:27.455721 6295 raft_consensus.cc:2781] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:27.455856 6295 raft_consensus.cc:686] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 1 LEADER]: Becoming Leader. State: Replica: ab031ddb50384fdb83abe63763b7d98a, State: Running, Role: LEADER
I20260924 00:14:27.455961 6295 consensus_queue.cc:237] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } }
I20260924 00:14:27.456700 5822 catalog_manager.cc:5495] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a reported cstate change: term changed from 0 to 1, leader changed from <none> to ab031ddb50384fdb83abe63763b7d98a (127.1.44.130). New cstate: current_term: 1 leader_uuid: "ab031ddb50384fdb83abe63763b7d98a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab031ddb50384fdb83abe63763b7d98a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:27.491685 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.490758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.1.44.130:43725)
0924 00:14:27.490802 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.490804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.490812 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:27.490878 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.490879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.490891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.490932 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.491468 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.491469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.491469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.491473 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.491486 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.491489 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.491535 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.491571 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.491582 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:14:27.493526 5965 raft_consensus.cc:1261] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 1 FOLLOWER]: Refusing update from remote peer ab031ddb50384fdb83abe63763b7d98a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:27.493535 6227 raft_consensus.cc:1261] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 1 FOLLOWER]: Refusing update from remote peer ab031ddb50384fdb83abe63763b7d98a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:27.493877 6295 consensus_queue.cc:1035] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1190e14133c241bb941a1cc755e9f3e2" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:27.494035 6297 consensus_queue.cc:1035] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36de5b678cb44030a2d7b3121ff81c58" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:27.496665 6301 mvcc.cc:205] Tried to move back new op lower bound from 7332695521250631680 to 7332695521100255232. Current Snapshot: MvccSnapshot[applied={T|T < 7332695521250631680}]
I20260924 00:14:27.497388 6302 mvcc.cc:205] Tried to move back new op lower bound from 7332695521250631680 to 7332695521100255232. Current Snapshot: MvccSnapshot[applied={T|T < 7332695521250631680}]
I20260924 00:14:27.498520 5823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:27.498616 5823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:27.499382 5823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 806344528faa4c5dacb02fee6fa46f06: Sending DeleteTablet for 3 replicas of tablet 1afe7df0af5b45d2a3ab32e50bcf2542
I20260924 00:14:27.499745 6207 tablet_service.cc:1514] Processing DeleteTablet for tablet 1afe7df0af5b45d2a3ab32e50bcf2542 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:27 UTC) from {username='slave'} at 127.0.0.1:53480
I20260924 00:14:27.499748 5945 tablet_service.cc:1514] Processing DeleteTablet for tablet 1afe7df0af5b45d2a3ab32e50bcf2542 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:27 UTC) from {username='slave'} at 127.0.0.1:32970
I20260924 00:14:27.499979 6311 tablet_replica.cc:330] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: stopping tablet replica
I20260924 00:14:27.499974 6076 tablet_service.cc:1514] Processing DeleteTablet for tablet 1afe7df0af5b45d2a3ab32e50bcf2542 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:27 UTC) from {username='slave'} at 127.0.0.1:59608
I20260924 00:14:27.500097 6311 raft_consensus.cc:2229] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:27.500209 6311 raft_consensus.cc:2258] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:27.500211 6312 tablet_replica.cc:330] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: stopping tablet replica
I20260924 00:14:27.500351 6312 raft_consensus.cc:2229] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:27.500389 6310 tablet_replica.cc:330] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58: stopping tablet replica
I20260924 00:14:27.500473 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5883
I20260924 00:14:27.500490 6310 raft_consensus.cc:2229] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:27.500579 6310 raft_consensus.cc:2258] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 36de5b678cb44030a2d7b3121ff81c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:27.501147 6312 raft_consensus.cc:2258] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:27.501636 6312 ts_tablet_manager.cc:1905] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:27.503685 6312 ts_tablet_manager.cc:1918] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:27.503808 6312 log.cc:1196] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/1afe7df0af5b45d2a3ab32e50bcf2542
I20260924 00:14:27.504149 6312 ts_tablet_manager.cc:1939] T 1afe7df0af5b45d2a3ab32e50bcf2542 P ab031ddb50384fdb83abe63763b7d98a: Deleting consensus metadata
W20260924 00:14:27.506141 5807 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:37833 (error 108)
I20260924 00:14:27.506325 5810 catalog_manager.cc:4841] TS ab031ddb50384fdb83abe63763b7d98a (127.1.44.130:43725): tablet 1afe7df0af5b45d2a3ab32e50bcf2542 (table test-table [id=bbbe2e0a0a044d8894fef939fc395e0c]) successfully deleted
I20260924 00:14:27.506681 6285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.506559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37833 (local address 127.0.0.1:32976)
0924 00:14:27.506605 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.506643 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:37833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 00:14:27.506858 5807 catalog_manager.cc:4584] TS 36de5b678cb44030a2d7b3121ff81c58 (127.1.44.129:37833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1afe7df0af5b45d2a3ab32e50bcf2542: Network error: Client connection negotiation failed: client connection to 127.1.44.129:37833: connect: Connection refused (error 111)
I20260924 00:14:27.507481 6311 ts_tablet_manager.cc:1905] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:27.507663 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6014
I20260924 00:14:27.509358 6311 ts_tablet_manager.cc:1918] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:27.509451 6311 log.cc:1196] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/1afe7df0af5b45d2a3ab32e50bcf2542
I20260924 00:14:27.509681 6311 ts_tablet_manager.cc:1939] T 1afe7df0af5b45d2a3ab32e50bcf2542 P 1190e14133c241bb941a1cc755e9f3e2: Deleting consensus metadata
I20260924 00:14:27.510053 5809 catalog_manager.cc:4841] TS 1190e14133c241bb941a1cc755e9f3e2 (127.1.44.131:39229): tablet 1afe7df0af5b45d2a3ab32e50bcf2542 (table test-table [id=bbbe2e0a0a044d8894fef939fc395e0c]) successfully deleted
I20260924 00:14:27.513347 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6145
I20260924 00:14:27.517642 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 5792
2026-09-24T00:14:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (718 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 00:14:27.536643 1202 mini_oidc.cc:152] Starting JWKS server
I20260924 00:14:27.537691 1202 webserver.cc:460] Webserver started at https://127.0.0.1:41279/ using document root <none> and password file <none>
I20260924 00:14:27.537791 1202 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 00:14:27.537955 1202 webserver.cc:460] Webserver started at http://127.0.0.1:38941/ using document root <none> and password file <none>
2026-09-24T00:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:27Z Disabled control of system clock
I20260924 00:14:27.545238 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:33077
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:33077
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/testchainca.cert with env {}
W20260924 00:14:27.623878 6319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:27.624068 6319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:27.624089 6319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:27.625268 6319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:27.625309 6319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:27.625322 6319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:27.625335 6319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:27.625347 6319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:14:27.625360 6319 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:14:27.625411 6319 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41279/jwks.json
I20260924 00:14:27.626871 6319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:33077
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:33077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:27.627108 6319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:27.627352 6319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:27.631954 6325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.632097 6319 server_base.cc:1034] running on GCE node
W20260924 00:14:27.632010 6327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.631954 6324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.632405 6319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:27.632647 6319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:27.633785 6319 hybrid_clock.cc:648] HybridClock initialized: now 1790208867633764 us; error 37 us; skew 500 ppm
I20260924 00:14:27.635017 6319 webserver.cc:460] Webserver started at http://127.1.44.190:44231/ using document root <none> and password file <none>
I20260924 00:14:27.635308 6319 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:27.635357 6319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:27.635499 6319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:27.636487 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "31fd311017ee4f9181f556780f1bc7b7"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "f6eb671b54a4c94d2fb1059d15aedfdd"
server_key_iv: "79560786310ffdede68a6e85da6d698a"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.636837 6319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "31fd311017ee4f9181f556780f1bc7b7"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "f6eb671b54a4c94d2fb1059d15aedfdd"
server_key_iv: "79560786310ffdede68a6e85da6d698a"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.638075 6319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:27.638767 6333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.638979 6319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:27.639047 6319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "31fd311017ee4f9181f556780f1bc7b7"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "f6eb671b54a4c94d2fb1059d15aedfdd"
server_key_iv: "79560786310ffdede68a6e85da6d698a"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.639137 6319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:27.649469 6319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:27.649739 6319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:27.649886 6319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:27.653718 6319 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:33077
I20260924 00:14:27.653748 6385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:33077 every 8 connection(s)
I20260924 00:14:27.654080 6319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:27.654960 6386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:27.657232 6386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7: Bootstrap starting.
I20260924 00:14:27.657819 6386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:27.658109 6386 log.cc:824] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:27.658747 6386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7: No bootstrap required, opened a new log
I20260924 00:14:27.659307 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6319
I20260924 00:14:27.659411 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:27.659590 1202 external_mini_cluster.cc:1428] Setting key dcc14d317e8ee367059b2fb73f84f5f7
I20260924 00:14:27.660851 6386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd311017ee4f9181f556780f1bc7b7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33077 } }
I20260924 00:14:27.661011 6386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:27.661047 6386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fd311017ee4f9181f556780f1bc7b7, State: Initialized, Role: FOLLOWER
I20260924 00:14:27.661180 6386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd311017ee4f9181f556780f1bc7b7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33077 } }
I20260924 00:14:27.661276 6386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:27.661341 6386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:27.661388 6386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:27.661912 6386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd311017ee4f9181f556780f1bc7b7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33077 } }
I20260924 00:14:27.661991 6386 leader_election.cc:302] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31fd311017ee4f9181f556780f1bc7b7; no voters:
I20260924 00:14:27.662164 6386 leader_election.cc:288] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:27.662197 6391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:27.662321 6391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [term 1 LEADER]: Becoming Leader. State: Replica: 31fd311017ee4f9181f556780f1bc7b7, State: Running, Role: LEADER
I20260924 00:14:27.662420 6386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:27.662403 6391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd311017ee4f9181f556780f1bc7b7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33077 } }
I20260924 00:14:27.662704 6391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31fd311017ee4f9181f556780f1bc7b7. Latest consensus state: current_term: 1 leader_uuid: "31fd311017ee4f9181f556780f1bc7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd311017ee4f9181f556780f1bc7b7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33077 } } }
I20260924 00:14:27.662765 6391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:27.662988 6392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31fd311017ee4f9181f556780f1bc7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd311017ee4f9181f556780f1bc7b7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33077 } } }
I20260924 00:14:27.663057 6392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:27.663349 6395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:27.663996 6395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:27.665795 6395 catalog_manager.cc:1347] Generated new cluster ID: 22540a752d224829bfbfada9e92be481
I20260924 00:14:27.665846 6395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:27.666298 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.660051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.1.44.190:33077)
0924 00:14:27.660258 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.660265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.660287 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:14:27.660831 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.660836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.660889 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.660999 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.661250 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.663125 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.664277 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.664793 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.664926 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:27.665598 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.665601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.665601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.665799 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.665839 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.665854 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.665922 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.666043 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.666116 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260924 00:14:27.667048 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:14:27.679802 6395 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:27.680474 6395 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:27.686030 6395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31fd311017ee4f9181f556780f1bc7b7: Generated new TSK 0
I20260924 00:14:27.686194 6395 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 00:14:27.715555 6416 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 00:14:27.715655 6416 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 00:14:27.715896 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.698214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34916 (local address 127.1.44.190:33077)
0924 00:14:27.698295 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.698297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.698408 (+ 111us) server_negotiation.cc:407] Connection header received
0924 00:14:27.698543 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.698544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.698556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.698652 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:14:27.698878 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.699729 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.701069 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.701404 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.701475 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:27.701786 (+ 311us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:14:27.715679 (+ 13893us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 00:14:27.715832 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260924 00:14:27.715938 6390 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 00:14:27.716411 6415 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:27.698139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33077 (local address 127.0.0.1:34916)
0924 00:14:27.698346 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.698358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.698430 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.698720 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.698721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.698723 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:14:27.698786 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:27.698789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.699825 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.699827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:27.700917 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:27.700920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.701489 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.701490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:27.701592 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:27.701636 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:14:27.716064 (+ 14428us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 00:14:27.716174 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.44.190:33077: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.44.190:33077: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 00:14:27.716872 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6319
2026-09-24T00:14:27Z chronyd exiting
I20260924 00:14:27.752772 1202 test_util.cc:175] -----------------------------------------------
I20260924 00:14:27.752892 1202 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790208846277299-1202-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (217 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 00:14:27.753615 1202 mini_oidc.cc:152] Starting JWKS server
I20260924 00:14:27.754289 1202 webserver.cc:460] Webserver started at https://127.0.0.1:33641/ using document root <none> and password file <none>
I20260924 00:14:27.754356 1202 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 00:14:27.754518 1202 webserver.cc:460] Webserver started at http://127.0.0.1:42871/ using document root <none> and password file <none>
2026-09-24T00:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:27Z Disabled control of system clock
I20260924 00:14:27.761871 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34787
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:37351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34787
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33641/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/testchainca.cert with env {}
W20260924 00:14:27.839970 6422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:27.840174 6422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:27.840196 6422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:27.841372 6422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:27.841416 6422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:27.841430 6422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:27.841475 6422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:27.841504 6422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:14:27.841528 6422 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:14:27.841552 6422 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33641/jwks.json
I20260924 00:14:27.843048 6422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33641/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:27.843323 6422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:27.843544 6422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:27.848181 6428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.848173 6430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:27.848183 6427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:27.848470 6422 server_base.cc:1034] running on GCE node
I20260924 00:14:27.848713 6422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:27.849010 6422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:27.850170 6422 hybrid_clock.cc:648] HybridClock initialized: now 1790208867850156 us; error 42 us; skew 500 ppm
I20260924 00:14:27.851397 6422 webserver.cc:460] Webserver started at http://127.1.44.190:37507/ using document root <none> and password file <none>
I20260924 00:14:27.851648 6422 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:27.851717 6422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:27.851857 6422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:27.852878 6422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "3f8d67b896394ab28ca167ce1bde4e84"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "54b32ab21fba90aa8d83d82cfc3e0019"
server_key_iv: "27ae83e656ea1db9235f94b66dabff59"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.853240 6422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "3f8d67b896394ab28ca167ce1bde4e84"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "54b32ab21fba90aa8d83d82cfc3e0019"
server_key_iv: "27ae83e656ea1db9235f94b66dabff59"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.854617 6422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:27.855436 6436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:27.856488 6422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:27.856568 6422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "3f8d67b896394ab28ca167ce1bde4e84"
format_stamp: "Formatted at 2026-09-24 00:14:27 on dist-test-slave-cm7b"
server_key: "54b32ab21fba90aa8d83d82cfc3e0019"
server_key_iv: "27ae83e656ea1db9235f94b66dabff59"
server_key_version: "encryptionkey@0"
I20260924 00:14:27.856621 6422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:27.868613 6422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:27.868952 6422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:27.869103 6422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:27.872797 6422 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34787
I20260924 00:14:27.872828 6488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34787 every 8 connection(s)
I20260924 00:14:27.873170 6422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:27.874053 6489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:27.875883 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6422
I20260924 00:14:27.875967 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:27.876143 1202 external_mini_cluster.cc:1428] Setting key 7e9900983590ba80a7a9f206d6142a33
I20260924 00:14:27.876525 6489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84: Bootstrap starting.
I20260924 00:14:27.877215 6489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:27.877525 6489 log.cc:824] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:27.878394 6489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84: No bootstrap required, opened a new log
I20260924 00:14:27.880554 6489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8d67b896394ab28ca167ce1bde4e84" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34787 } }
I20260924 00:14:27.880719 6489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:27.880764 6489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f8d67b896394ab28ca167ce1bde4e84, State: Initialized, Role: FOLLOWER
I20260924 00:14:27.880885 6489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8d67b896394ab28ca167ce1bde4e84" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34787 } }
I20260924 00:14:27.880967 6489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:27.881022 6489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:27.881071 6489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:27.881616 6489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8d67b896394ab28ca167ce1bde4e84" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34787 } }
I20260924 00:14:27.881727 6489 leader_election.cc:302] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f8d67b896394ab28ca167ce1bde4e84; no voters:
I20260924 00:14:27.881878 6489 leader_election.cc:288] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:27.881938 6494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:27.882089 6494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [term 1 LEADER]: Becoming Leader. State: Replica: 3f8d67b896394ab28ca167ce1bde4e84, State: Running, Role: LEADER
I20260924 00:14:27.882157 6489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:27.882179 6494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8d67b896394ab28ca167ce1bde4e84" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34787 } }
I20260924 00:14:27.882395 6492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.876656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37710 (local address 127.1.44.190:34787)
0924 00:14:27.876861 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.876871 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.876908 (+ 37us) server_negotiation.cc:407] Connection header received
0924 00:14:27.877523 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.877531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.877609 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.877758 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:27.877894 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.879182 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.880049 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.880696 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.880818 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:27.881565 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:27.881569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:27.881569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:27.881777 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:27.881819 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:27.881836 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:27.881904 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:27.882005 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 00:14:27.882175 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260924 00:14:27.882510 6494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f8d67b896394ab28ca167ce1bde4e84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8d67b896394ab28ca167ce1bde4e84" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34787 } } }
I20260924 00:14:27.882577 6494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:27.882683 6494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f8d67b896394ab28ca167ce1bde4e84. Latest consensus state: current_term: 1 leader_uuid: "3f8d67b896394ab28ca167ce1bde4e84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f8d67b896394ab28ca167ce1bde4e84" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34787 } } }
I20260924 00:14:27.882727 6494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:27.883343 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:14:27.883445 6499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:27.883937 6499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:27.885320 6499 catalog_manager.cc:1347] Generated new cluster ID: e4fa30dc40b143e68126bf0dd206ca14
I20260924 00:14:27.885377 6499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:27.893651 6499 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:27.894309 6499 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:27.899614 6499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f8d67b896394ab28ca167ce1bde4e84: Generated new TSK 0
I20260924 00:14:27.899772 6499 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 00:14:27.920033 6519 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 00:14:27.920125 6519 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 00:14:27.920470 6492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:27.906098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.1.44.190:34787)
0924 00:14:27.906159 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:27.906161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:27.906290 (+ 129us) server_negotiation.cc:407] Connection header received
0924 00:14:27.906359 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:27.906360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:27.906369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:27.906427 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:14:27.906568 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.907035 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.908183 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.908369 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.908421 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:27.908603 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:14:27.920278 (+ 11675us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33641/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 00:14:27.920405 (+ 127us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37724: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33641/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":36}
W20260924 00:14:27.920545 6492 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37724: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33641/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 00:14:27.920647 6518 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:27.906049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34787 (local address 127.0.0.1:37724)
0924 00:14:27.906216 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:27.906228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:27.906295 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:27.906445 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:27.906446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:27.906448 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:14:27.906507 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:27.906509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.907126 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.907127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:27.908105 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:27.908107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:27.908443 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:27.908444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:27.908503 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:27.908508 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:14:27.920402 (+ 11894us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33641/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 00:14:27.920530 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.44.190:34787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33641/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260924 00:14:27.920971 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6422
2026-09-24T00:14:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (204 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 00:14:27.958129 1202 mini_oidc.cc:152] Starting JWKS server
I20260924 00:14:27.958822 1202 webserver.cc:460] Webserver started at https://127.0.0.1:44581/ using document root <none> and password file <none>
I20260924 00:14:27.958891 1202 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 00:14:27.959069 1202 webserver.cc:460] Webserver started at http://127.0.0.1:35499/ using document root <none> and password file <none>
2026-09-24T00:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:27Z Disabled control of system clock
I20260924 00:14:27.966005 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34085
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34085
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44581/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 00:14:28.044261 6525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:28.044507 6525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:28.044543 6525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:28.045681 6525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:28.045751 6525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:28.045774 6525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:28.045796 6525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:28.045825 6525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:14:28.045848 6525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:14:28.045868 6525 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44581/jwks.json
I20260924 00:14:28.047246 6525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44581/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:28.047519 6525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:28.047797 6525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:28.052490 6531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:28.052518 6530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:28.052529 6525 server_base.cc:1034] running on GCE node
W20260924 00:14:28.052496 6533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:28.053004 6525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:28.053242 6525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:28.054421 6525 hybrid_clock.cc:648] HybridClock initialized: now 1790208868054406 us; error 37 us; skew 500 ppm
I20260924 00:14:28.055660 6525 webserver.cc:460] Webserver started at http://127.1.44.190:45785/ using document root <none> and password file <none>
I20260924 00:14:28.055910 6525 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:28.055956 6525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:28.056103 6525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:28.057121 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "e0d241e6592d46bfaeaea417f86bb6c1"
format_stamp: "Formatted at 2026-09-24 00:14:28 on dist-test-slave-cm7b"
server_key: "f5c301956ea6388e9d6a094362f7ed66"
server_key_iv: "e30b75de90985798aaab64ff82f4f146"
server_key_version: "encryptionkey@0"
I20260924 00:14:28.057473 6525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "e0d241e6592d46bfaeaea417f86bb6c1"
format_stamp: "Formatted at 2026-09-24 00:14:28 on dist-test-slave-cm7b"
server_key: "f5c301956ea6388e9d6a094362f7ed66"
server_key_iv: "e30b75de90985798aaab64ff82f4f146"
server_key_version: "encryptionkey@0"
I20260924 00:14:28.058765 6525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:28.059554 6539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:28.059764 6525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:28.059871 6525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "e0d241e6592d46bfaeaea417f86bb6c1"
format_stamp: "Formatted at 2026-09-24 00:14:28 on dist-test-slave-cm7b"
server_key: "f5c301956ea6388e9d6a094362f7ed66"
server_key_iv: "e30b75de90985798aaab64ff82f4f146"
server_key_version: "encryptionkey@0"
I20260924 00:14:28.059965 6525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:28.074097 6525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:28.074357 6525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:28.074512 6525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:28.078222 6525 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34085
I20260924 00:14:28.078253 6591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34085 every 8 connection(s)
I20260924 00:14:28.078572 6525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:28.079296 6592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:28.080029 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6525
I20260924 00:14:28.080137 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:28.080319 1202 external_mini_cluster.cc:1428] Setting key dfe92bbf448c12a4b740236948ddc74c
I20260924 00:14:28.082901 6592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1: Bootstrap starting.
I20260924 00:14:28.083616 6592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:28.083913 6592 log.cc:824] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:28.084673 6592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1: No bootstrap required, opened a new log
I20260924 00:14:28.085716 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:28.080794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.1.44.190:34085)
0924 00:14:28.080980 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:28.080987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:28.081020 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:14:28.081429 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:28.081435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:28.081491 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:28.081605 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:28.081845 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.082697 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.083805 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.084246 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.084368 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:28.085037 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:28.085042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:28.085042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:28.085240 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:28.085283 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:28.085296 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:28.085366 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:28.085424 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 00:14:28.085502 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260924 00:14:28.086728 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:14:28.087078 6592 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34085 } }
I20260924 00:14:28.087256 6592 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:28.087303 6592 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d241e6592d46bfaeaea417f86bb6c1, State: Initialized, Role: FOLLOWER
I20260924 00:14:28.087419 6592 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34085 } }
I20260924 00:14:28.087515 6592 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:28.087576 6592 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:28.087615 6592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:28.088172 6592 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34085 } }
I20260924 00:14:28.088255 6592 leader_election.cc:302] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0d241e6592d46bfaeaea417f86bb6c1; no voters:
I20260924 00:14:28.088369 6592 leader_election.cc:288] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:28.088449 6597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:28.088581 6597 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [term 1 LEADER]: Becoming Leader. State: Replica: e0d241e6592d46bfaeaea417f86bb6c1, State: Running, Role: LEADER
I20260924 00:14:28.088641 6592 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:28.088663 6597 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34085 } }
I20260924 00:14:28.089035 6599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0d241e6592d46bfaeaea417f86bb6c1. Latest consensus state: current_term: 1 leader_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34085 } } }
I20260924 00:14:28.089046 6598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d241e6592d46bfaeaea417f86bb6c1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34085 } } }
I20260924 00:14:28.089213 6599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:28.089232 6598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:28.089563 6607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:28.090049 6607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:28.091466 6607 catalog_manager.cc:1347] Generated new cluster ID: d43ed61721334363928bcd86d139b893
I20260924 00:14:28.091526 6607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:28.101249 6607 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:28.101745 6607 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:28.105029 6607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0d241e6592d46bfaeaea417f86bb6c1: Generated new TSK 0
I20260924 00:14:28.105185 6607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:28.114324 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:28.110754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.1.44.190:34085)
0924 00:14:28.110799 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:28.110801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:28.110881 (+ 80us) server_negotiation.cc:407] Connection header received
0924 00:14:28.110954 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:28.110954 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:28.110965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:28.111033 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:14:28.111172 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.111692 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.112837 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.113030 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.113078 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:28.113299 (+ 221us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:14:28.114102 (+ 803us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 00:14:28.114245 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 00:14:28.114387 6595 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 00:14:28.114421 6621 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:28.110635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34085 (local address 127.0.0.1:47418)
0924 00:14:28.110817 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:28.110829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:28.110895 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:28.111037 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:28.111037 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:28.111039 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:14:28.111104 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:28.111106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.111791 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.111792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:28.112751 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:28.112754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.113103 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.113104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:28.113172 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:28.113180 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:14:28.114221 (+ 1041us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 00:14:28.114307 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.44.190:34085: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44581/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260924 00:14:28.114634 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6525
E20260924 00:14:28.127071 6622 webserver.cc:533] Webserver: sslize failed:
E20260924 00:14:28.127141 6622 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T00:14:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (194 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T00:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:28Z Disabled control of system clock
I20260924 00:14:28.160303 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:40413
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:40413
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/testchainca.cert with env {}
W20260924 00:14:28.238099 6626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:28.238335 6626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:28.238372 6626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:28.239579 6626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:28.239637 6626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:28.239658 6626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:28.239681 6626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:28.239711 6626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 00:14:28.239735 6626 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 00:14:28.239764 6626 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 00:14:28.241214 6626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:40413
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:40413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:28.241496 6626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:28.241707 6626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:28.246356 6632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:28.246351 6631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:28.246429 6634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:28.246647 6626 server_base.cc:1034] running on GCE node
I20260924 00:14:28.246920 6626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:28.247260 6626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:28.248430 6626 hybrid_clock.cc:648] HybridClock initialized: now 1790208868248402 us; error 41 us; skew 500 ppm
I20260924 00:14:28.249573 6626 webserver.cc:460] Webserver started at http://127.1.44.190:42405/ using document root <none> and password file <none>
I20260924 00:14:28.249825 6626 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:28.249893 6626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:28.250017 6626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:28.251008 6626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "36e07ebc460747e58e87cb7805c67c29"
format_stamp: "Formatted at 2026-09-24 00:14:28 on dist-test-slave-cm7b"
server_key: "3f150cfcd0ee664d646d5fb9553433c7"
server_key_iv: "e5ac56ab122a66291a5affc506ed311a"
server_key_version: "encryptionkey@0"
I20260924 00:14:28.251389 6626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "36e07ebc460747e58e87cb7805c67c29"
format_stamp: "Formatted at 2026-09-24 00:14:28 on dist-test-slave-cm7b"
server_key: "3f150cfcd0ee664d646d5fb9553433c7"
server_key_iv: "e5ac56ab122a66291a5affc506ed311a"
server_key_version: "encryptionkey@0"
I20260924 00:14:28.252652 6626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:28.253405 6640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:28.253597 6626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:28.253701 6626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "36e07ebc460747e58e87cb7805c67c29"
format_stamp: "Formatted at 2026-09-24 00:14:28 on dist-test-slave-cm7b"
server_key: "3f150cfcd0ee664d646d5fb9553433c7"
server_key_iv: "e5ac56ab122a66291a5affc506ed311a"
server_key_version: "encryptionkey@0"
I20260924 00:14:28.253804 6626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:28.261463 6626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:28.261716 6626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:28.261858 6626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:28.265648 6626 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:40413
I20260924 00:14:28.265677 6692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:40413 every 8 connection(s)
I20260924 00:14:28.265983 6626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:28.266860 6693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:28.268965 6693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29: Bootstrap starting.
I20260924 00:14:28.269448 6693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:28.269682 6693 log.cc:824] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:28.270289 6693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29: No bootstrap required, opened a new log
I20260924 00:14:28.271852 6693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e07ebc460747e58e87cb7805c67c29" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 40413 } }
I20260924 00:14:28.271992 6693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:28.272014 6693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e07ebc460747e58e87cb7805c67c29, State: Initialized, Role: FOLLOWER
I20260924 00:14:28.272136 6693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e07ebc460747e58e87cb7805c67c29" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 40413 } }
I20260924 00:14:28.272210 6693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:28.272231 6693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:28.272258 6693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:28.272706 6693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e07ebc460747e58e87cb7805c67c29" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 40413 } }
I20260924 00:14:28.272782 6693 leader_election.cc:302] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36e07ebc460747e58e87cb7805c67c29; no voters:
I20260924 00:14:28.272895 6693 leader_election.cc:288] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:28.272970 6696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:28.273106 6696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [term 1 LEADER]: Becoming Leader. State: Replica: 36e07ebc460747e58e87cb7805c67c29, State: Running, Role: LEADER
I20260924 00:14:28.273149 6693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:28.273198 6696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e07ebc460747e58e87cb7805c67c29" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 40413 } }
I20260924 00:14:28.273543 6698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36e07ebc460747e58e87cb7805c67c29. Latest consensus state: current_term: 1 leader_uuid: "36e07ebc460747e58e87cb7805c67c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e07ebc460747e58e87cb7805c67c29" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 40413 } } }
I20260924 00:14:28.273623 6698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:28.273854 6703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:28.273893 6697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36e07ebc460747e58e87cb7805c67c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36e07ebc460747e58e87cb7805c67c29" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 40413 } } }
I20260924 00:14:28.273960 6697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:28.274219 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6626
I20260924 00:14:28.274333 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:28.274431 6703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:28.274521 1202 external_mini_cluster.cc:1428] Setting key 153f26d6fac44c674e4775937f1e19ed
I20260924 00:14:28.276186 6703 catalog_manager.cc:1347] Generated new cluster ID: 7948e8e6bb304e9db34ea229a5e133a1
I20260924 00:14:28.276238 6703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:28.280700 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:28.274921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.1.44.190:40413)
0924 00:14:28.275352 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:28.275358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:28.275530 (+ 172us) server_negotiation.cc:407] Connection header received
0924 00:14:28.275761 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:28.275767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:28.275875 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:28.276020 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:28.276253 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.277581 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.278665 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.279270 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.279372 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:28.280043 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:28.280046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:28.280047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:28.280232 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:28.280268 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:28.280281 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:28.280339 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:28.280437 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 00:14:28.280514 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":71,"threads_started":1}
I20260924 00:14:28.281455 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:14:28.285029 6703 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:28.285483 6703 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:28.293563 6703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36e07ebc460747e58e87cb7805c67c29: Generated new TSK 0
I20260924 00:14:28.293731 6703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:28.392848 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:28.301623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.1.44.190:40413)
0924 00:14:28.301688 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:28.301690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:28.301733 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:14:28.301790 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:28.301791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:28.301803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:28.301885 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0924 00:14:28.301996 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.302823 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.303630 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.303956 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.304027 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:28.304169 (+ 142us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 00:14:28.392559 (+ 88390us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 00:14:28.392764 (+ 205us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51798: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260924 00:14:28.392943 6715 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51798: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 00:14:28.392920 6722 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:28.301520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:40413 (local address 127.0.0.1:51798)
0924 00:14:28.301671 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:28.301681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:28.301735 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:28.301881 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:28.301882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:28.301883 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0924 00:14:28.301926 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:28.301927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.302900 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.302900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:28.303546 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:28.303549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:28.304031 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:28.304032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:28.304069 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:28.304074 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 00:14:28.392739 (+ 88665us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 00:14:28.392842 (+ 103us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.44.190:40413: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260924 00:14:28.393307 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6626
2026-09-24T00:14:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (257 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:28 dist-test-slave-cm7b krb5kdc[6734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:28 dist-test-slave-cm7b krb5kdc[6734](info): set up 2 sockets
Sep 24 00:14:28 dist-test-slave-cm7b krb5kdc[6734](info): commencing operation
krb5kdc: starting...
W20260924 00:14:29.587383 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:29 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:29.700279 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42627
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:42627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:29.779534 6747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:29.779748 6747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:29.779772 6747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:29.780865 6747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:29.780898 6747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:29.780912 6747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:29.780925 6747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:29.780937 6747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:29.782298 6747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:42627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:29.782477 6747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:29.782649 6747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:29.787303 6752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:29.787309 6753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:29.787321 6755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:29.787487 6747 server_base.cc:1034] running on GCE node
I20260924 00:14:29.787679 6747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:29.787909 6747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:29.789049 6747 hybrid_clock.cc:648] HybridClock initialized: now 1790208869789031 us; error 36 us; skew 500 ppm
Sep 24 00:14:29 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208869, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:29.794768 6747 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:29.795149 6747 webserver.cc:460] Webserver started at http://127.1.44.190:40653/ using document root <none> and password file <none>
I20260924 00:14:29.795395 6747 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:29.795444 6747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:29.795532 6747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:29.796521 6747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "3ca3640a779146d49612ea5d13576485"
format_stamp: "Formatted at 2026-09-24 00:14:29 on dist-test-slave-cm7b"
server_key: "b9d8b2db03e1ae57dba806c2a8713596"
server_key_iv: "dc5f47a1cf2862aa3eaef38c824828f2"
server_key_version: "encryptionkey@0"
I20260924 00:14:29.796838 6747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "3ca3640a779146d49612ea5d13576485"
format_stamp: "Formatted at 2026-09-24 00:14:29 on dist-test-slave-cm7b"
server_key: "b9d8b2db03e1ae57dba806c2a8713596"
server_key_iv: "dc5f47a1cf2862aa3eaef38c824828f2"
server_key_version: "encryptionkey@0"
I20260924 00:14:29.798036 6747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:29.798687 6762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:29.798853 6747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:29.798957 6747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "3ca3640a779146d49612ea5d13576485"
format_stamp: "Formatted at 2026-09-24 00:14:29 on dist-test-slave-cm7b"
server_key: "b9d8b2db03e1ae57dba806c2a8713596"
server_key_iv: "dc5f47a1cf2862aa3eaef38c824828f2"
server_key_version: "encryptionkey@0"
I20260924 00:14:29.799028 6747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:29.813558 6747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:29.815584 6747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:29.815737 6747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:29.819476 6747 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:42627
I20260924 00:14:29.819509 6814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:42627 every 8 connection(s)
I20260924 00:14:29.819851 6747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:29.820676 6815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:29.822845 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485: Bootstrap starting.
I20260924 00:14:29.823472 6815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:29.823764 6815 log.cc:824] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:29.824406 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485: No bootstrap required, opened a new log
I20260924 00:14:29.824972 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6747
I20260924 00:14:29.825088 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:29.825260 1202 external_mini_cluster.cc:1428] Setting key 93f298f129cb847df1822ce8825b1fbc
I20260924 00:14:29.826308 6815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca3640a779146d49612ea5d13576485" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42627 } }
I20260924 00:14:29.826478 6815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:29.826517 6815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca3640a779146d49612ea5d13576485, State: Initialized, Role: FOLLOWER
I20260924 00:14:29.826659 6815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca3640a779146d49612ea5d13576485" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42627 } }
I20260924 00:14:29.826745 6815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:29.826783 6815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:29.826836 6815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:29.827800 6815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca3640a779146d49612ea5d13576485" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42627 } }
I20260924 00:14:29.827935 6815 leader_election.cc:302] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ca3640a779146d49612ea5d13576485; no voters:
I20260924 00:14:29.828099 6815 leader_election.cc:288] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:29.828150 6820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:29.828281 6820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [term 1 LEADER]: Becoming Leader. State: Replica: 3ca3640a779146d49612ea5d13576485, State: Running, Role: LEADER
I20260924 00:14:29.828358 6815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:29.828369 6820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca3640a779146d49612ea5d13576485" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42627 } }
I20260924 00:14:29.828848 6820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ca3640a779146d49612ea5d13576485. Latest consensus state: current_term: 1 leader_uuid: "3ca3640a779146d49612ea5d13576485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca3640a779146d49612ea5d13576485" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42627 } } }
I20260924 00:14:29.828923 6820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:29.829077 6821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ca3640a779146d49612ea5d13576485" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca3640a779146d49612ea5d13576485" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42627 } } }
I20260924 00:14:29.829128 6821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:29.829210 6828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:29.829677 6828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:29.831342 6828 catalog_manager.cc:1347] Generated new cluster ID: 556519f7d3d04670b14c793fd8362985
I20260924 00:14:29.831393 6828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:29 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:29.836768 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:29.825762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.1.44.190:42627)
0924 00:14:29.825921 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:29.825928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:29.825952 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:14:29.826291 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:29.826297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:29.826340 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:29.826448 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:29.827278 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:29.828318 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:29.829421 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:29.830138 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:29.830244 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:29.833032 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:29.833035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:29.833036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:29.833047 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:29.834969 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:29.835380 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:29.835390 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:29.835391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:29.835511 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:29.835719 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:29.835720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:29.835721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:29.836028 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:29.836195 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:29.836268 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 00:14:29.836416 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260924 00:14:29.842087 6828 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:29.842588 6828 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:29.844501 6828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ca3640a779146d49612ea5d13576485: Generated new TSK 0
I20260924 00:14:29.844657 6828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:29.872229 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42627
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:29.951676 6841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:29.951872 6841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:29.951894 6841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:29.952957 6841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:29.952992 6841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:29.953018 6841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:29.954411 6841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:29.954593 6841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:29.954758 6841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:29.959668 6847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:29.959672 6841 server_base.cc:1034] running on GCE node
W20260924 00:14:29.959618 6846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:29.959630 6849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:29.960041 6841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:29.960248 6841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:29.961382 6841 hybrid_clock.cc:648] HybridClock initialized: now 1790208869961362 us; error 36 us; skew 500 ppm
Sep 24 00:14:29 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208869, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:29.963871 6841 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:29.964279 6841 webserver.cc:460] Webserver started at http://127.1.44.129:42861/ using document root <none> and password file <none>
I20260924 00:14:29.964521 6841 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:29.964572 6841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:29.964751 6841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:29.965782 6841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "91f697fd828d4ba5a61c44a012b357a0"
format_stamp: "Formatted at 2026-09-24 00:14:29 on dist-test-slave-cm7b"
server_key: "acb90809a1ca25820af0b96f697c2077"
server_key_iv: "a8737a3786f690838cf3325a4e7c3cc6"
server_key_version: "encryptionkey@0"
I20260924 00:14:29.966130 6841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "91f697fd828d4ba5a61c44a012b357a0"
format_stamp: "Formatted at 2026-09-24 00:14:29 on dist-test-slave-cm7b"
server_key: "acb90809a1ca25820af0b96f697c2077"
server_key_iv: "a8737a3786f690838cf3325a4e7c3cc6"
server_key_version: "encryptionkey@0"
I20260924 00:14:29.967427 6841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:14:29.968154 6856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:29.968333 6841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:29.968420 6841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "91f697fd828d4ba5a61c44a012b357a0"
format_stamp: "Formatted at 2026-09-24 00:14:29 on dist-test-slave-cm7b"
server_key: "acb90809a1ca25820af0b96f697c2077"
server_key_iv: "a8737a3786f690838cf3325a4e7c3cc6"
server_key_version: "encryptionkey@0"
I20260924 00:14:29.968499 6841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:29.978696 6841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:29.980511 6841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:29.980667 6841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:29.980892 6841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:29.981175 6841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:29.981227 6841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:29.981267 6841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:29.981295 6841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:29.986910 6841 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:38185
I20260924 00:14:29.986944 6969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:38185 every 8 connection(s)
I20260924 00:14:29.987272 6841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:14:29 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208869, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:29.996663 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6841
I20260924 00:14:29.996675 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:29.987978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:56987 (local address 127.1.44.190:42627)
0924 00:14:29.988022 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:29.988024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:29.988195 (+ 171us) server_negotiation.cc:407] Connection header received
0924 00:14:29.989159 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:29.989160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:29.989177 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:29.989222 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:29.989874 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:29.990381 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:29.991452 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:29.991671 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:29.991756 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:29.993827 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:29.993829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:29.993829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:29.993839 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:29.995306 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:29.995704 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:29.995705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:29.995705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:29.995765 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:29.996009 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:29.996010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:29.996011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:29.996124 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:29.996200 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:29.996470 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 00:14:29.996583 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:29.996791 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:29.996829 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:29.987940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42627 (local address 127.1.44.129:56987)
0924 00:14:29.988071 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:29.988116 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:29.989046 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:29.989299 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:29.989303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:29.989584 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:29.989767 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:29.989772 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:29.990491 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:29.990492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:29.991356 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:29.991358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:29.991764 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:29.991764 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:29.991890 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:29.992398 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:29.992405 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:29.993728 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:29.995397 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:29.995404 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:29.995406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:29.995627 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:29.995839 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:29.995840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:29.995841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:29.995944 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:29.996191 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:29.996197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:29.996334 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 00:14:29.996435 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:14:29.996599 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260924 00:14:29.996999 1202 external_mini_cluster.cc:1428] Setting key 869322238be00fa820da934543560a5d
I20260924 00:14:29.997519 6970 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42627
I20260924 00:14:29.997644 6970 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:29.997807 6970 heartbeater.cc:511] Master 127.1.44.190:42627 requested a full tablet report, sending...
I20260924 00:14:29.998363 6779 ts_manager.cc:194] Registered new tserver with Master: 91f697fd828d4ba5a61c44a012b357a0 (127.1.44.129:38185)
I20260924 00:14:29.999226 6779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:56987
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:30.021713 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42627
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:30.101203 6975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:30.101397 6975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:30.101419 6975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:30.102605 6975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:30.102650 6975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:30.102675 6975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:30.104163 6975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:30.104393 6975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:30.104624 6975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:30.109644 6981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:30.109653 6983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:30.109652 6980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:30.109938 6975 server_base.cc:1034] running on GCE node
I20260924 00:14:30.110154 6975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:30.110397 6975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:30.111555 6975 hybrid_clock.cc:648] HybridClock initialized: now 1790208870111549 us; error 35 us; skew 500 ppm
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.114071 6975 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:30.114493 6975 webserver.cc:460] Webserver started at http://127.1.44.130:35483/ using document root <none> and password file <none>
I20260924 00:14:30.114720 6975 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:30.114789 6975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:30.114936 6975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:30.116091 6975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "80865f47e31246fe9944a19514b27ab7"
format_stamp: "Formatted at 2026-09-24 00:14:30 on dist-test-slave-cm7b"
server_key: "dff2c7eab04a0971b1505132cdb1ed2e"
server_key_iv: "4bdbf7148647757870df7057193ee720"
server_key_version: "encryptionkey@0"
I20260924 00:14:30.116441 6975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "80865f47e31246fe9944a19514b27ab7"
format_stamp: "Formatted at 2026-09-24 00:14:30 on dist-test-slave-cm7b"
server_key: "dff2c7eab04a0971b1505132cdb1ed2e"
server_key_iv: "4bdbf7148647757870df7057193ee720"
server_key_version: "encryptionkey@0"
I20260924 00:14:30.117717 6975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:14:30.118384 6990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.118597 6975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:30.118693 6975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "80865f47e31246fe9944a19514b27ab7"
format_stamp: "Formatted at 2026-09-24 00:14:30 on dist-test-slave-cm7b"
server_key: "dff2c7eab04a0971b1505132cdb1ed2e"
server_key_iv: "4bdbf7148647757870df7057193ee720"
server_key_version: "encryptionkey@0"
I20260924 00:14:30.118747 6975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:30.157017 6975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:30.159435 6975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:30.159544 6975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:30.159749 6975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:30.160084 6975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:30.160121 6975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.160166 6975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:30.160197 6975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.165844 6975 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:36129
I20260924 00:14:30.165886 7103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:36129 every 8 connection(s)
I20260924 00:14:30.166246 6975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:30.166637 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 6975
I20260924 00:14:30.166736 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:30.166947 1202 external_mini_cluster.cc:1428] Setting key f5d8edc09a60235b9b7a7b18e79bc704
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
I20260924 00:14:30.176309 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.167034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:51415 (local address 127.1.44.190:42627)
0924 00:14:30.167083 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.167085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.167347 (+ 262us) server_negotiation.cc:407] Connection header received
0924 00:14:30.168273 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.168274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.168286 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.168331 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.168955 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.169486 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.170476 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.170791 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.170862 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.173095 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:30.173097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:30.173098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:30.173106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:30.174844 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.175301 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.175303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.175303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.175367 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.175617 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.175618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.175618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.175774 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:30.175845 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:30.176092 (+ 247us) server_negotiation.cc:299] Negotiation successful
0924 00:14:30.176184 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:30.176452 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.167036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42627 (local address 127.1.44.130:51415)
0924 00:14:30.167208 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:30.167249 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:30.168165 (+ 916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:30.168399 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:30.168403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:30.168664 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:30.168857 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:30.168861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.169578 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.169579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:30.170365 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:30.170367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.170876 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.170877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:30.170985 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.171555 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:30.171563 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:30.172987 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:30.174945 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:30.174951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:30.174952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:30.175181 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:30.175434 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:30.175434 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:30.175434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:30.175531 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:30.175839 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:30.175843 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:30.175957 (+ 114us) client_negotiation.cc:769] Sending connection context
0924 00:14:30.176034 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 00:14:30.176162 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260924 00:14:30.176909 7104 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42627
I20260924 00:14:30.177006 7104 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:30.177148 7104 heartbeater.cc:511] Master 127.1.44.190:42627 requested a full tablet report, sending...
I20260924 00:14:30.177525 6779 ts_manager.cc:194] Registered new tserver with Master: 80865f47e31246fe9944a19514b27ab7 (127.1.44.130:36129)
I20260924 00:14:30.177903 6779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:51415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:30.192368 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42627
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:30.271709 7109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:30.271901 7109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:30.271924 7109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:30.273066 7109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:30.273108 7109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:30.273133 7109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:30.274497 7109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:30.274668 7109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:30.274829 7109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:30.279531 7115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:30.279531 7117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:30.279547 7114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:30.279807 7109 server_base.cc:1034] running on GCE node
I20260924 00:14:30.279996 7109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:30.280215 7109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:30.281350 7109 hybrid_clock.cc:648] HybridClock initialized: now 1790208870281338 us; error 27 us; skew 500 ppm
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.283815 7109 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:30.284243 7109 webserver.cc:460] Webserver started at http://127.1.44.131:36797/ using document root <none> and password file <none>
I20260924 00:14:30.284413 7109 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:30.284461 7109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:30.284547 7109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:30.285490 7109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "8442fd3d73a24b498e56c69d5a922736"
format_stamp: "Formatted at 2026-09-24 00:14:30 on dist-test-slave-cm7b"
server_key: "f372c5bca08164a6e6f8fa75694da722"
server_key_iv: "2dfcf173ae3a9e5d636a529296a1aff3"
server_key_version: "encryptionkey@0"
I20260924 00:14:30.285777 7109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "8442fd3d73a24b498e56c69d5a922736"
format_stamp: "Formatted at 2026-09-24 00:14:30 on dist-test-slave-cm7b"
server_key: "f372c5bca08164a6e6f8fa75694da722"
server_key_iv: "2dfcf173ae3a9e5d636a529296a1aff3"
server_key_version: "encryptionkey@0"
I20260924 00:14:30.286948 7109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:30.287640 7124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.287806 7109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.287902 7109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "8442fd3d73a24b498e56c69d5a922736"
format_stamp: "Formatted at 2026-09-24 00:14:30 on dist-test-slave-cm7b"
server_key: "f372c5bca08164a6e6f8fa75694da722"
server_key_iv: "2dfcf173ae3a9e5d636a529296a1aff3"
server_key_version: "encryptionkey@0"
I20260924 00:14:30.287979 7109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:30.338186 7109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:30.340353 7109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:30.340523 7109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:30.340771 7109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:30.341099 7109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:30.341154 7109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.341197 7109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:30.341231 7109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:30.347163 7109 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:39487
I20260924 00:14:30.347249 7237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:39487 every 8 connection(s)
I20260924 00:14:30.347589 7109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:30.356837 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.348328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:48853 (local address 127.1.44.190:42627)
0924 00:14:30.348413 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.348416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.348595 (+ 179us) server_negotiation.cc:407] Connection header received
0924 00:14:30.349503 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.349504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.349514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.349591 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.350498 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.350995 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.352006 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.352183 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.352238 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.354244 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:30.354245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:30.354245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:30.354251 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:30.355653 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.356027 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.356028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.356029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.356078 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.356305 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.356306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.356307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.356440 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:30.356492 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:30.356717 (+ 225us) server_negotiation.cc:299] Negotiation successful
0924 00:14:30.356773 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 00:14:30.356999 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.348292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42627 (local address 127.1.44.131:48853)
0924 00:14:30.348474 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:30.348508 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:30.349408 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:30.349667 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:30.349671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:30.350070 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:30.350371 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:30.350377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.351079 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.351080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:30.351934 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:30.351936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.352256 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.352257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:30.352351 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.352865 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:30.352872 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:30.354159 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:30.355733 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:30.355738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:30.355740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:30.355952 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:30.356136 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:30.356137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:30.356137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:30.356229 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:30.356495 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:30.356499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:30.356609 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 00:14:30.356689 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 00:14:30.356800 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260924 00:14:30.357481 7238 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42627
I20260924 00:14:30.357496 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7109
I20260924 00:14:30.357594 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:30.357594 7238 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:30.357769 1202 external_mini_cluster.cc:1428] Setting key d958ef968aab4e8cccd2d05f43678d08
I20260924 00:14:30.357767 7238 heartbeater.cc:511] Master 127.1.44.190:42627 requested a full tablet report, sending...
I20260924 00:14:30.358260 6778 ts_manager.cc:194] Registered new tserver with Master: 8442fd3d73a24b498e56c69d5a922736 (127.1.44.131:39487)
I20260924 00:14:30.358747 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:14:30.358769 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:48853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/test-admin.keytab.
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.373174 1202 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 00:14:30.374733 7245 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:30.373995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.1.44.129:38185)
0924 00:14:30.374171 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.374176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.374185 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:30.374210 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.374211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.374224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.374287 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.374649 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58364: BlockingRecv error: recv got EOF from 127.0.0.1:58364 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
W20260924 00:14:30.374898 1202 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc' of type FILE
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.375885 1202 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.376817 1202 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:30.384321 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.377327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.1.44.129:38185)
0924 00:14:30.377381 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.377382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.377390 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:30.377453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.377454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.377464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.377508 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.377860 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.378558 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.379250 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.379564 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.379618 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.381557 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:30.381565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:30.381566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:30.381573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:30.383126 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.383525 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.383528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.383528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.383584 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.383798 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.383799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.383800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.383923 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:30.384068 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:30.384140 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 00:14:30.384214 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260924 00:14:30.386031 7245 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:30.385478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.1.44.129:38185)
0924 00:14:30.385514 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.385516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.385552 (+ 36us) server_negotiation.cc:407] Connection header received
0924 00:14:30.385630 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.385631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.385641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.385689 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.385984 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58380: BlockingRecv error: recv got EOF from 127.0.0.1:58380 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 00:14:30.386124 1202 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc' of type FILE
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.387044 1202 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.387923 1202 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:30.394083 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.388388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.1.44.129:38185)
0924 00:14:30.388435 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.388436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.388448 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:14:30.388492 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.388493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.388499 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.388545 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.388873 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.389340 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.389970 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.390155 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.390243 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.392142 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:30.392143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:30.392143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:30.392149 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:30.393270 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.393563 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.393563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.393564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.393608 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.393749 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.393750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.393750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.393823 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:30.393884 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:30.393974 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 00:14:30.394024 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 00:14:30.395675 7245 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:30.395018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58398 (local address 127.1.44.129:38185)
0924 00:14:30.395061 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.395063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.395132 (+ 69us) server_negotiation.cc:407] Connection header received
0924 00:14:30.395238 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.395239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.395249 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.395306 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.395622 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58398: BlockingRecv error: recv got EOF from 127.0.0.1:58398 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260924 00:14:30.395803 1202 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790208846277299-1202-0/krb5kdc/krb5cc' of type FILE
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.396684 1202 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:30.397461 1202 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[6734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.129@KRBTEST.COM
I20260924 00:14:30.404142 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:30.397884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58402 (local address 127.1.44.129:38185)
0924 00:14:30.397948 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:30.397950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:30.398037 (+ 87us) server_negotiation.cc:407] Connection header received
0924 00:14:30.398098 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:30.398099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:30.398108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:30.398161 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:30.398512 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.398949 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.399719 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:30.399910 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:30.399996 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:30.401898 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:30.401899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:30.401899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:30.401906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:30.403074 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.403445 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.403446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.403446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.403496 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:30.403691 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:30.403692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:30.403692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:30.403784 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:30.403880 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:30.403986 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 00:14:30.404053 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:30.404796 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6841
I20260924 00:14:30.409608 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6975
I20260924 00:14:30.414111 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7109
I20260924 00:14:30.418286 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 6747
2026-09-24T00:14:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1853 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[7258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[7258](info): set up 2 sockets
Sep 24 00:14:30 dist-test-slave-cm7b krb5kdc[7258](info): commencing operation
krb5kdc: starting...
W20260924 00:14:31.451469 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:31 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab.
Entry for principal oryx/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab.
I20260924 00:14:31.566107 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37919
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab
--principal=oryx/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:31.646948 7271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:31.647169 7271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:31.647209 7271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:31.648346 7271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:31.648414 7271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:31.648440 7271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:31.648473 7271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:31.648497 7271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:31.649966 7271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab
--principal=oryx/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:31.650242 7271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:31.650453 7271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:31.655128 7277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:31.655265 7276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:31.655334 7279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:31.655171 7271 server_base.cc:1034] running on GCE node
I20260924 00:14:31.655602 7271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:31.655920 7271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:31.657056 7271 hybrid_clock.cc:648] HybridClock initialized: now 1790208871657042 us; error 41 us; skew 500 ppm
Sep 24 00:14:31 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:31.659436 7271 init.cc:377] Logged in from keytab as oryx/127.1.44.190@KRBTEST.COM (short username oryx)
I20260924 00:14:31.659894 7271 webserver.cc:460] Webserver started at http://127.1.44.190:32903/ using document root <none> and password file <none>
I20260924 00:14:31.660178 7271 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:31.660247 7271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:31.660379 7271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:31.661370 7271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "1fd90ec71d534cf096671d64b6337386"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "0082367bba1e43a598abcd56bffabace"
server_key_iv: "2a1313ffd616b6f831566fd71f6be0f3"
server_key_version: "encryptionkey@0"
I20260924 00:14:31.661713 7271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "1fd90ec71d534cf096671d64b6337386"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "0082367bba1e43a598abcd56bffabace"
server_key_iv: "2a1313ffd616b6f831566fd71f6be0f3"
server_key_version: "encryptionkey@0"
I20260924 00:14:31.662992 7271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:14:31.663782 7286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:31.663941 7271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:31.664064 7271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "1fd90ec71d534cf096671d64b6337386"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "0082367bba1e43a598abcd56bffabace"
server_key_iv: "2a1313ffd616b6f831566fd71f6be0f3"
server_key_version: "encryptionkey@0"
I20260924 00:14:31.664153 7271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:31.687984 7271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:31.689801 7271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:31.689960 7271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:31.693941 7271 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:37919
I20260924 00:14:31.693974 7338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:37919 every 8 connection(s)
I20260924 00:14:31.694311 7271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:31.695328 7339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:31.698810 7339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Bootstrap starting.
I20260924 00:14:31.699558 7339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:31.699873 7339 log.cc:824] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:31.700691 7339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: No bootstrap required, opened a new log
I20260924 00:14:31.700749 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7271
I20260924 00:14:31.701287 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:31.701534 1202 external_mini_cluster.cc:1428] Setting key 2aa81c519034698fb281e77c95d090e4
I20260924 00:14:31.703171 7339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } }
I20260924 00:14:31.703377 7339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:31.703425 7339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd90ec71d534cf096671d64b6337386, State: Initialized, Role: FOLLOWER
I20260924 00:14:31.703563 7339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } }
I20260924 00:14:31.703644 7339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:31.703683 7339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:31.703740 7339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:31.704874 7339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } }
I20260924 00:14:31.704977 7339 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fd90ec71d534cf096671d64b6337386; no voters:
I20260924 00:14:31.705116 7339 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:31.705255 7344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:31.705332 7339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:31.705456 7344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 LEADER]: Becoming Leader. State: Replica: 1fd90ec71d534cf096671d64b6337386, State: Running, Role: LEADER
I20260924 00:14:31.705622 7344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } }
I20260924 00:14:31.705963 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd90ec71d534cf096671d64b6337386. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } }
I20260924 00:14:31.706060 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:31.705971 7345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } }
I20260924 00:14:31.706157 7345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:31.706869 7353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:31.707484 7353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:31.708974 7353 catalog_manager.cc:1347] Generated new cluster ID: 7ff6785193474eb4b423f2110201643c
I20260924 00:14:31.709029 7353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:14:31 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:31.713340 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:31.702026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.1.44.190:37919)
0924 00:14:31.702194 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:31.702204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:31.702236 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:14:31.703153 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:31.703163 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:31.703248 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:31.703451 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:31.704262 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:31.705671 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:31.706839 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:31.707413 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:31.707527 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:31.709991 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:31.709995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:31.709995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:31.710006 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:31.711854 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:31.712210 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:31.712213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:31.712214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:31.712323 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:31.712528 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:31.712529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:31.712530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:31.712767 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:31.712947 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:31.713009 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 00:14:31.713146 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260924 00:14:31.736246 7353 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:31.736977 7353 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:31.751308 7353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Generated new TSK 0
I20260924 00:14:31.751518 7353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab.
Entry for principal oryx/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab.
I20260924 00:14:31.780022 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37919
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:31.869901 7365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:31.870105 7365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:31.870127 7365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:31.871292 7365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:31.871332 7365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:31.871359 7365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:31.872763 7365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:31.872987 7365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:31.873243 7365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:14:31.878116 7365 server_base.cc:1034] running on GCE node
W20260924 00:14:31.878085 7373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:31.878096 7371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:31.878088 7370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:31.878479 7365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:31.878720 7365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:31.879889 7365 hybrid_clock.cc:648] HybridClock initialized: now 1790208871879870 us; error 42 us; skew 500 ppm
Sep 24 00:14:31 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:31.882284 7365 init.cc:377] Logged in from keytab as oryx/127.1.44.129@KRBTEST.COM (short username oryx)
I20260924 00:14:31.882738 7365 webserver.cc:460] Webserver started at http://127.1.44.129:35489/ using document root <none> and password file <none>
I20260924 00:14:31.882954 7365 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:31.883019 7365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:31.883167 7365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:31.884186 7365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "830acc9966fa1cbf1bb807747ae426c7"
server_key_iv: "60985ce05ce5f990d483476a35259547"
server_key_version: "encryptionkey@0"
I20260924 00:14:31.884547 7365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "830acc9966fa1cbf1bb807747ae426c7"
server_key_iv: "60985ce05ce5f990d483476a35259547"
server_key_version: "encryptionkey@0"
I20260924 00:14:31.885738 7365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:14:31.886387 7380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:31.886579 7365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:31.886662 7365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "830acc9966fa1cbf1bb807747ae426c7"
server_key_iv: "60985ce05ce5f990d483476a35259547"
server_key_version: "encryptionkey@0"
I20260924 00:14:31.886711 7365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:31.900754 7365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:31.902925 7365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:31.903033 7365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:31.903265 7365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:31.903581 7365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:31.903616 7365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:31.903672 7365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:31.903697 7365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:31.909960 7365 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:44703
I20260924 00:14:31.910005 7493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:44703 every 8 connection(s)
I20260924 00:14:31.910298 7365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:31.914907 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7365
I20260924 00:14:31.915046 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:31.915302 1202 external_mini_cluster.cc:1428] Setting key a920e6b34cd0369531922d5e50ce0ced
Sep 24 00:14:31 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:31.920028 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:31.910995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:33895 (local address 127.1.44.190:37919)
0924 00:14:31.911060 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:31.911062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:31.911204 (+ 142us) server_negotiation.cc:407] Connection header received
0924 00:14:31.911771 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:31.911772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:31.911784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:31.911842 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:31.912715 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:31.913201 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:31.914147 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:31.914357 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:31.914422 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:31.916770 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:31.916772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:31.916772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:31.916779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:31.918412 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:31.918925 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:31.918926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:31.918926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:31.918993 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:31.919306 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:31.919307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:31.919307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:31.919416 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:31.919478 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:31.919878 (+ 400us) server_negotiation.cc:299] Negotiation successful
0924 00:14:31.919964 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:14:31.920234 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:31.910931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:33895)
0924 00:14:31.911078 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:31.911118 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:31.911670 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:31.911894 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:31.911898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:31.912433 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:31.912626 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:31.912630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:31.913279 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:31.913280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:31.914050 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:31.914052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:31.914436 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:31.914436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:31.914543 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:31.915088 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:31.915099 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:31.916669 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:31.918509 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:31.918516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:31.918518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:31.918826 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:31.919069 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:31.919070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:31.919071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:31.919242 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:31.919491 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:31.919497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:31.919671 (+ 174us) client_negotiation.cc:769] Sending connection context
0924 00:14:31.919787 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 00:14:31.919965 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260924 00:14:31.920739 7494 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37919
I20260924 00:14:31.920848 7494 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:31.921021 7494 heartbeater.cc:511] Master 127.1.44.190:37919 requested a full tablet report, sending...
I20260924 00:14:31.921538 7303 ts_manager.cc:194] Registered new tserver with Master: 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703)
I20260924 00:14:31.922137 7303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:33895
WARNING: no policy specified for oryx/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab.
Entry for principal oryx/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab.
I20260924 00:14:31.940284 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37919
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:32.019282 7499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:32.019477 7499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:32.019500 7499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:32.020633 7499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:32.020673 7499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:32.020699 7499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:32.022111 7499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:32.022354 7499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:32.022588 7499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.027699 7505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.027714 7507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.027721 7504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.027743 7499 server_base.cc:1034] running on GCE node
I20260924 00:14:32.028174 7499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.028384 7499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.029562 7499 hybrid_clock.cc:648] HybridClock initialized: now 1790208872029548 us; error 28 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.032034 7499 init.cc:377] Logged in from keytab as oryx/127.1.44.130@KRBTEST.COM (short username oryx)
I20260924 00:14:32.032501 7499 webserver.cc:460] Webserver started at http://127.1.44.130:42387/ using document root <none> and password file <none>
I20260924 00:14:32.032733 7499 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.032799 7499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.032949 7499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:32.033926 7499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "f61e8ac142799e099437e5fa550a1760"
server_key_iv: "e7be1f1d1d4e71b27cbec8641d14196b"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.034281 7499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "f61e8ac142799e099437e5fa550a1760"
server_key_iv: "e7be1f1d1d4e71b27cbec8641d14196b"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.035607 7499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:32.036355 7514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.036535 7499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:32.036645 7499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "f61e8ac142799e099437e5fa550a1760"
server_key_iv: "e7be1f1d1d4e71b27cbec8641d14196b"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.036720 7499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.049124 7499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.050951 7499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:32.051105 7499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.051362 7499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:32.051652 7499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:32.051704 7499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.051744 7499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:32.051771 7499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.057579 7499 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:44951
I20260924 00:14:32.057627 7627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:44951 every 8 connection(s)
I20260924 00:14:32.057909 7499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:32.065143 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7499
I20260924 00:14:32.065268 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:32.065470 1202 external_mini_cluster.cc:1428] Setting key dc34a0eb6853b423be1dcfd07f203d4a
I20260924 00:14:32.067188 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.058568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:57165 (local address 127.1.44.190:37919)
0924 00:14:32.058611 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.058613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.058853 (+ 240us) server_negotiation.cc:407] Connection header received
0924 00:14:32.059413 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.059414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.059426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.059483 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.060420 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.060898 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.061858 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.062051 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.062108 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.064196 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.064197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.064198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.064204 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.065559 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.066021 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.066022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.066023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.066083 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.066516 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.066516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.066516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.066615 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.066689 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.067004 (+ 315us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.067071 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:32.067467 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.058543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:57165)
0924 00:14:32.058699 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.058739 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.059325 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.059553 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.059556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.060125 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.060322 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.060327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.060983 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.060984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.061771 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.061773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.062137 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.062137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.062240 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.062770 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.062778 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.064116 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.065633 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.065640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.065641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.065932 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.066298 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.066299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.066300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.066449 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.066699 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.066705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.066883 (+ 178us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.066991 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.067182 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260924 00:14:32.067855 7628 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37919
I20260924 00:14:32.067961 7628 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:32.068154 7628 heartbeater.cc:511] Master 127.1.44.190:37919 requested a full tablet report, sending...
I20260924 00:14:32.068521 7303 ts_manager.cc:194] Registered new tserver with Master: 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951)
I20260924 00:14:32.068892 7303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:57165
WARNING: no policy specified for oryx/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab.
Entry for principal oryx/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab.
I20260924 00:14:32.090660 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37919
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:32.169553 7633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:32.169744 7633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:32.169764 7633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:32.170871 7633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:32.170913 7633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:32.170984 7633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:32.172456 7633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:32.172688 7633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:32.172907 7633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.177994 7638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.178030 7633 server_base.cc:1034] running on GCE node
W20260924 00:14:32.178004 7639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.178000 7641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.178462 7633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.178686 7633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.179836 7633 hybrid_clock.cc:648] HybridClock initialized: now 1790208872179821 us; error 25 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.182248 7633 init.cc:377] Logged in from keytab as oryx/127.1.44.131@KRBTEST.COM (short username oryx)
I20260924 00:14:32.182677 7633 webserver.cc:460] Webserver started at http://127.1.44.131:43921/ using document root <none> and password file <none>
I20260924 00:14:32.182847 7633 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.182885 7633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.182971 7633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:32.183991 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "161e7e7cf01244ea85e124b73c88d69c"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "db4790588fb4c3f81b493128a49e1d08"
server_key_iv: "9d9e62b1e3433ec75feb0aae26f7a1f9"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.184295 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "161e7e7cf01244ea85e124b73c88d69c"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "db4790588fb4c3f81b493128a49e1d08"
server_key_iv: "9d9e62b1e3433ec75feb0aae26f7a1f9"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.185571 7633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:14:32.186225 7648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.186379 7633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:32.186458 7633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "161e7e7cf01244ea85e124b73c88d69c"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "db4790588fb4c3f81b493128a49e1d08"
server_key_iv: "9d9e62b1e3433ec75feb0aae26f7a1f9"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.186508 7633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.201540 7633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.203644 7633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:32.203804 7633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.204027 7633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:32.204329 7633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:32.204381 7633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.204417 7633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:32.204445 7633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.210158 7633 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:44661
I20260924 00:14:32.210227 7761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:44661 every 8 connection(s)
I20260924 00:14:32.210543 7633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:32.214900 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7633
I20260924 00:14:32.215001 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:32.215229 1202 external_mini_cluster.cc:1428] Setting key f16dba72a59ee9d231631b028eb43722
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:32.219280 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.211243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:36261 (local address 127.1.44.190:37919)
0924 00:14:32.211310 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.211312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.211554 (+ 242us) server_negotiation.cc:407] Connection header received
0924 00:14:32.212009 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.212010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.212021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.212073 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.213046 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.213545 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.214504 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.214675 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.214726 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.216978 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.216980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.216980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.216986 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.218099 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.218429 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.218430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.218430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.218476 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.218724 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.218725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.218725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.218817 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.218865 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.219101 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.219158 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:14:32.219506 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.211228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:36261)
0924 00:14:32.211442 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.211476 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.211930 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.212139 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.212142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.212757 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.212944 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.212950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.213633 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.213633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.214435 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.214437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.214757 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.214758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.214873 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.215439 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.215448 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.216902 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.218163 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.218168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.218170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.218367 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.218552 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.218553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.218553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.218664 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.218871 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.218874 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.218993 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.219075 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.219188 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260924 00:14:32.219956 7762 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37919
I20260924 00:14:32.220052 7762 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:32.220227 7762 heartbeater.cc:511] Master 127.1.44.190:37919 requested a full tablet report, sending...
I20260924 00:14:32.220513 7303 ts_manager.cc:194] Registered new tserver with Master: 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661)
I20260924 00:14:32.220890 7303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:36261
I20260924 00:14:32.227059 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.44.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.189.keytab.
Entry for principal oryx/127.1.44.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.189.keytab.
I20260924 00:14:32.251556 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:45337
--webserver_interface=127.1.44.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.189.keytab
--principal=oryx/127.1.44.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37919,127.1.44.189:45337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:32.331046 7767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:32.331290 7767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:32.331315 7767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:32.332396 7767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:32.332432 7767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:32.332448 7767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:32.332459 7767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:32.332471 7767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:32.333801 7767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37919,127.1.44.189:45337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.189.keytab
--principal=oryx/127.1.44.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:45337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.44.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
W20260924 00:14:32.333890 7767 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.44.190:37919,127.1.44.189:45337'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 00:14:32.333998 7767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:32.334146 7767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.338639 7772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.338718 7775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.338634 7773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.338902 7767 server_base.cc:1034] running on GCE node
I20260924 00:14:32.339181 7767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.339515 7767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.340658 7767 hybrid_clock.cc:648] HybridClock initialized: now 1790208872340636 us; error 37 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.343115 7767 init.cc:377] Logged in from keytab as oryx/127.1.44.189@KRBTEST.COM (short username oryx)
I20260924 00:14:32.343544 7767 webserver.cc:460] Webserver started at http://127.1.44.189:45171/ using document root <none> and password file <none>
I20260924 00:14:32.343766 7767 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.343832 7767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.343976 7767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:32.345011 7767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/instance:
uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "004f79465f6ad5d4303938df6dbda4b9"
server_key_iv: "4b20c0258942f2e3e19d5d6f182825b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.345366 7767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal/instance:
uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "004f79465f6ad5d4303938df6dbda4b9"
server_key_iv: "4b20c0258942f2e3e19d5d6f182825b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.346669 7767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:14:32.347384 7782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.347561 7767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:32.347654 7767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "004f79465f6ad5d4303938df6dbda4b9"
server_key_iv: "4b20c0258942f2e3e19d5d6f182825b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.347733 7767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.357220 7767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.358851 7767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:32.358997 7767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.362643 7767 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.189:45337
I20260924 00:14:32.362689 7834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.189:45337 every 8 connection(s)
I20260924 00:14:32.363005 7767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
I20260924 00:14:32.363984 7835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:32.365372 7835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:14:32.366050 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7767
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.189@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
I20260924 00:14:32.374506 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.366032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.1.44.190:37919)
0924 00:14:32.366096 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.366098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.366319 (+ 221us) server_negotiation.cc:407] Connection header received
0924 00:14:32.366925 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.366926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.366937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.366979 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.367575 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.368095 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.369008 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.369198 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.369282 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.371486 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.371487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.371487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.371493 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.373101 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.373462 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.373463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.373463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.373521 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.373741 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.373742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.373743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.373936 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.373991 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.374329 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.374419 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:14:32.374650 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.366019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45100)
0924 00:14:32.366191 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.366238 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.366816 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.366998 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.367002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.367334 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.367479 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.367484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.368194 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.368195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.368897 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.368898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.369289 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.369290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.369407 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.369909 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.369919 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.371406 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.373179 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.373184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.373186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.373379 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.373591 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.373592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.373592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.373678 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.374011 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.374017 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.374189 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.374303 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.374438 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260924 00:14:32.375268 7835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:14:32.375409 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.366677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51980 (local address 127.1.44.189:45337)
0924 00:14:32.366827 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.366831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.366863 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:14:32.366947 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.366948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.366968 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.367046 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.367729 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.368543 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.369275 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.369580 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.369656 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.372077 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.372085 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.372085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.372091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.373782 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.374169 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.374172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.374172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.374230 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.374765 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.374766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.374766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.374920 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.375062 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.375136 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.375231 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.189@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
I20260924 00:14:32.380976 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.375558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:51986)
0924 00:14:32.375593 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.375604 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.375644 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.375752 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.375753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.375892 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.375919 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.375921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.376523 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.376524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.376980 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.376981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.377355 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.377357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.377407 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.377832 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.377837 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.379028 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.380271 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.380272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.380272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.380426 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.380573 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.380573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.380573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.380612 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.380754 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.380755 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.380784 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.380827 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.380887 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 00:14:32.381050 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.375632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.1.44.189:45337)
0924 00:14:32.375660 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.375662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.375669 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:32.375687 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.375687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.375697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.375749 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.376000 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.376435 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.377077 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.377277 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.377357 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.379101 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.379102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.379103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.379108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.380206 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.380487 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.380488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.380488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.380534 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.380645 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.380645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.380646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.380721 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.380762 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.380842 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.380929 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":14}
I20260924 00:14:32.382201 7835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Bootstrap starting.
I20260924 00:14:32.382937 7835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:32.383153 7835 log.cc:824] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:32.383829 7835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: No bootstrap required, opened a new log
I20260924 00:14:32.385085 7835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } }
I20260924 00:14:32.385190 7835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:32.385212 7835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eaaaf9c3dbe440693c907ffec519dd3, State: Initialized, Role: FOLLOWER
I20260924 00:14:32.385324 7835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } }
I20260924 00:14:32.385545 7841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } } }
I20260924 00:14:32.385627 7841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:14:32.385639 7835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 00:14:32.386996 7852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:14:32.387063 7852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:14:32.388650 7767 master_runner.cc:428] Detected that this master 6eaaaf9c3dbe440693c907ffec519dd3 is joining an existing cluster
I20260924 00:14:32.388695 7767 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.44.189:45337
I20260924 00:14:32.396104 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.389795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.1.44.189:45337)
0924 00:14:32.389910 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.389912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.389918 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:14:32.389940 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.389940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.389956 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.390001 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.390553 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.391077 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.391983 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.392268 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.392334 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.393263 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.393265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.393265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.393272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.394810 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.395162 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.395164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.395164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.395255 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.395475 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.395475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.395476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.395600 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.395672 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.395843 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.395985 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":99}
I20260924 00:14:32.396216 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.389699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:51996)
0924 00:14:32.389835 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.389846 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.389897 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.390008 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.390008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.390170 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.390474 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.390476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.391152 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.391153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.391889 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.391891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.392389 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.392389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.392445 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.392820 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.392825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.393181 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.394895 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.394896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.394896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.395085 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.395335 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.395337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.395337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.395400 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.395676 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.395677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.395718 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.395931 (+ 213us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.396087 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":30,"thread_start_us":83,"threads_started":1}
I20260924 00:14:32.397537 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.389727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.1.44.190:37919)
0924 00:14:32.389881 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.389883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.390279 (+ 396us) server_negotiation.cc:407] Connection header received
0924 00:14:32.390344 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.390345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.390354 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.390416 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.390772 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.391524 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.392327 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.392530 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.392852 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.393961 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.393962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.393963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.394151 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.395723 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.396785 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.396786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.396786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.396845 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.397035 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.397036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.397037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.397148 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.397222 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.397379 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.397453 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260924 00:14:32.397826 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.390101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45104)
0924 00:14:32.390228 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.390237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.390288 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.390431 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.390432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.390626 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.390683 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.390684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.391615 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.391616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.392250 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.392251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.392855 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.392855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.392921 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.393500 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.393505 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.393859 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.395793 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.395794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.395794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.396094 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.396923 (+ 829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.396923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.396923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.396969 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.397214 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.397214 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.397265 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.397352 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.397455 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.190@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
I20260924 00:14:32.406983 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.399869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.1.44.189:45337)
0924 00:14:32.399911 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.399912 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.400019 (+ 107us) server_negotiation.cc:407] Connection header received
0924 00:14:32.400096 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.400097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.400104 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.400143 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.400599 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.401015 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.402085 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.402250 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.402303 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.404242 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.404244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.404244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.404251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.405713 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.406212 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.406213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.406214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.406270 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.406467 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.406468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.406468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.406575 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.406649 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.406824 (+ 175us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.406898 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:32.407025 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.399792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:52006)
0924 00:14:32.399914 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.399952 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.400025 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.400158 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.400161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.400469 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.400522 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.400524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.401087 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.401088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.402006 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.402008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.402322 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.402323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.402399 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.402785 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.402790 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.404143 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.405840 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.405846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.405848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.406124 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.406356 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.406356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.406357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.406416 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.406642 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.406649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.406698 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.406799 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.406914 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260924 00:14:32.407565 7303 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.44.189:45337: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
member_type: NON_VOTER
last_known_addr {
host: "127.1.44.189"
port: 45337
}
attrs {
promote: true
}
}
dest_uuid: "1fd90ec71d534cf096671d64b6337386"
cas_config_opid_index: -1
I20260924 00:14:32.407876 7303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } }
I20260924 00:14:32.408349 7861 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.408569 7809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:32.409416 7861 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.44.189:45337
I20260924 00:14:32.409462 7809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Refusing update from remote peer 1fd90ec71d534cf096671d64b6337386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 00:14:32.409495 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.409581 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:32.409767 7862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:32.409861 7767 master.cc:537] Master@127.1.44.189:45337 shutting down...
W20260924 00:14:32.410454 7287 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260924 00:14:32.410845 7287 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 -> Peer 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189:45337): Couldn't send request to peer 6eaaaf9c3dbe440693c907ffec519dd3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:14:32.413003 7767 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:32.413115 7767 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:32.413168 7767 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: stopping tablet replica
I20260924 00:14:32.425601 7767 master.cc:552] Master@127.1.44.189:45337 shutdown complete.
I20260924 00:14:32.425657 7767 master_runner.cc:305] Clearing existing system tablet
I20260924 00:14:32.425787 7767 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.425834 7767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.426543 7767 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:32.427045 7866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.427179 7767 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.427275 7767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.427300 7767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "004f79465f6ad5d4303938df6dbda4b9"
server_key_iv: "4b20c0258942f2e3e19d5d6f182825b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.428061 7767 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:32.429682 7767 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 00:14:32.429749 7767 log.cc:1196] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 00:14:32.429940 7767 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Deleting consensus metadata
I20260924 00:14:32.430059 7767 master_runner.cc:315] Copying system tablet from 127.1.44.190:37919
I20260924 00:14:32.430640 7767 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: tablet copy: Beginning tablet copy session from remote peer at address 127.1.44.190:37919
I20260924 00:14:32.436830 7872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.430940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45118)
0924 00:14:32.431044 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.431056 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.431113 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.431270 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.431270 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.431429 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.431481 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.431483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.432406 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.432406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.432935 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.432936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.433405 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.433406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.433453 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.433923 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.433928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.434197 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.435944 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.435945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.435945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.436101 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.436294 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.436294 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.436294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.436342 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.436597 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.436598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.436634 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.436688 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.436759 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":67,"threads_started":1}
I20260924 00:14:32.436889 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.431006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.1.44.190:37919)
0924 00:14:32.431057 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.431059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.431103 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:14:32.431176 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.431177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.431187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.431260 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.431558 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.432322 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.433029 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.433332 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.433397 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.434278 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.434280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.434280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.434287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.435854 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.436171 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.436172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.436173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.436230 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.436402 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.436403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.436403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.436536 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.436602 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.436728 (+ 126us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.436799 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 00:14:32.437330 7323 tablet_copy_service.cc:140] P 1fd90ec71d534cf096671d64b6337386: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6eaaaf9c3dbe440693c907ffec519dd3 ({username='oryx', principal='oryx/127.1.44.189@KRBTEST.COM'} at 127.0.0.1:45118)
I20260924 00:14:32.437438 7323 tablet_copy_service.cc:161] P 1fd90ec71d534cf096671d64b6337386: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6eaaaf9c3dbe440693c907ffec519dd3 at {username='oryx', principal='oryx/127.1.44.189@KRBTEST.COM'} at 127.0.0.1:45118: session id = 6eaaaf9c3dbe440693c907ffec519dd3-00000000000000000000000000000000
I20260924 00:14:32.438036 7323 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 00:14:32.438632 7767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 00:14:32.439795 7767 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: tablet copy: Starting download of 0 data blocks...
I20260924 00:14:32.439939 7767 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: tablet copy: Starting download of 1 WAL segments...
I20260924 00:14:32.440688 7767 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 00:14:32.441484 7323 tablet_copy_service.cc:342] P 1fd90ec71d534cf096671d64b6337386: Request end of tablet copy session 6eaaaf9c3dbe440693c907ffec519dd3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.44.189@KRBTEST.COM'} at 127.0.0.1:45118
I20260924 00:14:32.441529 7323 tablet_copy_service.cc:433] P 1fd90ec71d534cf096671d64b6337386: ending tablet copy session 6eaaaf9c3dbe440693c907ffec519dd3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6eaaaf9c3dbe440693c907ffec519dd3
I20260924 00:14:32.443962 7767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.445115 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.445180 7873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.445241 7874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.445400 7767 server_base.cc:1034] running on GCE node
I20260924 00:14:32.445504 7767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.445654 7767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.446777 7767 hybrid_clock.cc:648] HybridClock initialized: now 1790208872446769 us; error 27 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.448159 7767 init.cc:377] Logged in from keytab as oryx/127.1.44.189@KRBTEST.COM (short username oryx)
I20260924 00:14:32.448441 7767 webserver.cc:460] Webserver started at http://127.1.44.189:46691/ using document root <none> and password file <none>
I20260924 00:14:32.448558 7767 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.448612 7767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.449195 7767 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:32.449684 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.449846 7767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:32.449903 7767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "004f79465f6ad5d4303938df6dbda4b9"
server_key_iv: "4b20c0258942f2e3e19d5d6f182825b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.449955 7767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.458498 7767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.460060 7767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.462956 7767 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.189:45337
I20260924 00:14:32.463352 7935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.189:45337 every 8 connection(s)
I20260924 00:14:32.463361 7767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
I20260924 00:14:32.464089 7936 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:14:32.464466 7936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Bootstrap starting.
I20260924 00:14:32.466689 7936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:14:32.466907 7936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: Bootstrap complete.
I20260924 00:14:32.467093 7936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } }
I20260924 00:14:32.467172 7936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6eaaaf9c3dbe440693c907ffec519dd3, State: Initialized, Role: LEARNER
I20260924 00:14:32.467231 7936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } }
I20260924 00:14:32.467399 7936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:32.467394 7939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.467499 7939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:14:32.468784 7950 catalog_manager.cc:1259] Loaded cluster ID: 7ff6785193474eb4b423f2110201643c
I20260924 00:14:32.468946 7950 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: loading cluster ID for follower catalog manager: success
I20260924 00:14:32.470041 7950 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: acquiring CA information for follower catalog manager: success
I20260924 00:14:32.470328 7950 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:14:32.482970 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.476774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.1.44.189:45337)
0924 00:14:32.476883 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.476885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.476893 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:32.476919 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.476920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.476928 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.476995 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.477503 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.478107 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.478816 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.479026 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.479112 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.480375 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.480376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.480377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.480383 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.481816 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.482186 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.482187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.482188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.482247 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.482460 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.482461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.482462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.482565 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.482665 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.482775 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.482853 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
WARNING: no policy specified for oryx/127.1.44.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.188.keytab.
Entry for principal oryx/127.1.44.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.188.keytab.
I20260924 00:14:32.510959 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:45671
--webserver_interface=127.1.44.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.188.keytab
--principal=oryx/127.1.44.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:32.596719 7954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:32.596925 7954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:32.596946 7954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:32.597995 7954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:32.598028 7954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:32.598042 7954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:32.598054 7954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:32.598066 7954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:32.599521 7954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.188.keytab
--principal=oryx/127.1.44.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:45671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.44.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:32.599695 7954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:32.599858 7954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.604509 7962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.604518 7954 server_base.cc:1034] running on GCE node
W20260924 00:14:32.604482 7959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.604468 7960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.605020 7954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.605252 7954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.606456 7954 hybrid_clock.cc:648] HybridClock initialized: now 1790208872606430 us; error 42 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.608866 7954 init.cc:377] Logged in from keytab as oryx/127.1.44.188@KRBTEST.COM (short username oryx)
I20260924 00:14:32.609301 7954 webserver.cc:460] Webserver started at http://127.1.44.188:39763/ using document root <none> and password file <none>
I20260924 00:14:32.609535 7954 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.609601 7954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.609747 7954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:32.610747 7954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/instance:
uuid: "b002ba0cbd174921bc293680923e5a71"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "d8b1f1b475d5bf7502660f5a589a80d6"
server_key_iv: "1ff3aa3dfadac52b6e00a7c42a3acc38"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.611096 7954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal/instance:
uuid: "b002ba0cbd174921bc293680923e5a71"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "d8b1f1b475d5bf7502660f5a589a80d6"
server_key_iv: "1ff3aa3dfadac52b6e00a7c42a3acc38"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.612387 7954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:32.613109 7969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.613308 7954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:32.613404 7954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
uuid: "b002ba0cbd174921bc293680923e5a71"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "d8b1f1b475d5bf7502660f5a589a80d6"
server_key_iv: "1ff3aa3dfadac52b6e00a7c42a3acc38"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.613488 7954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.628647 7954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.630388 7954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:32.630548 7954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.635572 7954 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.188:45671
I20260924 00:14:32.635651 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.188:45671 every 8 connection(s)
I20260924 00:14:32.636024 7954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
I20260924 00:14:32.636942 8022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:32.638785 8022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.188@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:32.646139 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 7954
I20260924 00:14:32.647400 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.639638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.1.44.190:37919)
0924 00:14:32.639682 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.639684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.639787 (+ 103us) server_negotiation.cc:407] Connection header received
0924 00:14:32.640222 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.640223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.640231 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.640290 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.640921 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.641395 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.642330 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.642585 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.642640 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.644656 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.644657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.644657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.644663 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.645803 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.646280 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.646282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.646282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.646349 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.646628 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.646629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.646630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.646784 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.646848 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.647266 (+ 418us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.647333 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:14:32.647614 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.639511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45130)
0924 00:14:32.639664 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.639704 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.640143 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.640339 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.640342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.640655 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.640844 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.640848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.641474 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.641475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.642247 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.642248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.642659 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.642659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.642752 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.643253 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.643262 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.644576 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.645869 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.645874 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.645875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.646180 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.646420 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.646422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.646422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.646536 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.646883 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.646887 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.647067 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.647218 (+ 151us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.647386 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260924 00:14:32.648113 8022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.188@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.44.188@KRBTEST.COM
I20260924 00:14:32.655393 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.648348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:52022)
0924 00:14:32.648408 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.648416 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.648446 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.648566 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.648567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.648716 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.648760 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.648761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.649685 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.649686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.650472 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.650474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.650911 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.650912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.650981 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.651387 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.651391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.652531 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.654370 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.654371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.654371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.654621 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.654811 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.654812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.654812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.654872 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.655101 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.655101 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.655146 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.655242 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.655322 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 00:14:32.655462 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.648402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.1.44.189:45337)
0924 00:14:32.648441 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.648443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.648452 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:32.648495 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.648496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.648506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.648549 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.648835 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.649595 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.650561 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.650845 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.650907 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.652615 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.652617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.652617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.652626 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.654286 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.654701 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.654702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.654702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.654750 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.654935 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.654935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.654935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.655028 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.655087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.655270 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.655360 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:32.655804 8022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:14:32.656474 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.646649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.1.44.188:45671)
0924 00:14:32.646804 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.646808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.646826 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:14:32.646846 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.646849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.646861 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.646930 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.647770 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.648789 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.649811 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.650246 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.650314 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.653367 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.653378 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.653379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.653384 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.655221 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.655608 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.655611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.655611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.655670 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.655856 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.655857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.655858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.656053 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.656231 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.656303 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.656363 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":100,"threads_started":1}
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.188@KRBTEST.COM for oryx/127.1.44.188@KRBTEST.COM
I20260924 00:14:32.662190 8024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.656051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38508)
0924 00:14:32.656097 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.656107 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.656140 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.656578 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.656579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.656757 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.656785 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.656786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.657462 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.657463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.657907 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.657908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.658230 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.658231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.658267 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.658559 (+ 292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.658563 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.659647 (+ 1084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.661437 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.661438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.661438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.661587 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.661749 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.661750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.661750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.661803 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.661981 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.661982 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.662011 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.662069 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.662135 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 00:14:32.662343 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.656097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.1.44.188:45671)
0924 00:14:32.656453 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.656455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.656461 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:14:32.656474 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.656475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.656482 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.656556 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.656871 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.657402 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.657971 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.658171 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.658217 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.659721 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.659722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.659722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.659727 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.661376 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.661648 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.661649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.661649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.661695 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.661852 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.661852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.661852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.661932 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.661975 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.662082 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.662176 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":341,"thread_start_us":76,"threads_started":1}
I20260924 00:14:32.663646 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Bootstrap starting.
I20260924 00:14:32.664234 8022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:32.664528 8022 log.cc:824] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:32.665252 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: No bootstrap required, opened a new log
I20260924 00:14:32.666685 8022 raft_consensus.cc:348] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } }
I20260924 00:14:32.666846 8022 raft_consensus.cc:374] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:32.666891 8022 raft_consensus.cc:729] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b002ba0cbd174921bc293680923e5a71, State: Initialized, Role: FOLLOWER
I20260924 00:14:32.667024 8022 consensus_queue.cc:260] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } }
I20260924 00:14:32.667325 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } } }
I20260924 00:14:32.667384 8022 sys_catalog.cc:564] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:32.667440 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 00:14:32.669183 8039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:14:32.669274 8039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:14:32.671214 7954 master_runner.cc:428] Detected that this master b002ba0cbd174921bc293680923e5a71 is joining an existing cluster
I20260924 00:14:32.671260 7954 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.44.188:45671
I20260924 00:14:32.679026 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.672271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.1.44.189:45337)
0924 00:14:32.672336 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.672338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.672348 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:32.672437 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.672438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.672447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.672499 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.672946 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.673507 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.674619 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.674904 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.674955 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.676153 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.676155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.676155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.676162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.677779 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.678165 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.678166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.678166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.678217 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.678459 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.678460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.678460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.678586 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.678628 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.678831 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.678911 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:32.679462 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.672125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:52024)
0924 00:14:32.672269 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.672278 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.672350 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.672547 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.672548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.672772 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.672846 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.672848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.673629 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.673629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.674531 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.674533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.675047 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.675048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.675119 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.675626 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.675630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.676050 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.677867 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.677868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.677869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.678090 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.678289 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.678290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.678291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.678363 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.678676 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.678677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.678718 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.679260 (+ 542us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.679352 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260924 00:14:32.700460 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.672445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.1.44.190:37919)
0924 00:14:32.672487 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.672488 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.672739 (+ 251us) server_negotiation.cc:407] Connection header received
0924 00:14:32.672786 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.672787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.672797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.672855 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.673716 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.674460 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.675563 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.675855 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.675930 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.677217 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.677219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.677219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.677226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.678965 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.699589 (+ 20624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.699591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.699592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.699686 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.699925 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.699926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.699927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.700057 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.700139 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.700293 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.700370 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:32.700542 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.672402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45144)
0924 00:14:32.672671 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.672680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.672732 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.672901 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.672902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.673127 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.673629 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.673630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.674552 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.674552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.675450 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.675452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.675983 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.675984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.676050 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.676739 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.676744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.677115 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.679041 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.679042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.679042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.699484 (+ 20442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.699762 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.699763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.699764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.699841 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.700128 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.700129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.700173 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.700256 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.700382 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":47,"thread_start_us":60,"threads_started":1}
I20260924 00:14:32.700763 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.672600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.1.44.188:45671)
0924 00:14:32.672633 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.672635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.672906 (+ 271us) server_negotiation.cc:407] Connection header received
0924 00:14:32.672931 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.672932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.672939 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.672978 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.673789 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.674483 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.675981 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.676707 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.676771 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.678372 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.678373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.678373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.678378 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.680054 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.699683 (+ 19629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.699684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.699684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.699737 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.699962 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.699963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.699964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.700093 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.700152 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.700344 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.700685 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":196,"server-negotiator.queue_time_us":19}
I20260924 00:14:32.700905 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.672566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38510)
0924 00:14:32.672814 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.672823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.672861 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.673008 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.673008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.673650 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.673706 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.673708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.674579 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.674580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.675898 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.675901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.676814 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.676815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.676870 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.677665 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.677670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.678287 (+ 617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.680131 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.680132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.680132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.699592 (+ 19460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.699811 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.699812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.699812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.699906 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.700174 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.700175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.700217 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.700273 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.700450 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":113,"thread_start_us":51,"threads_started":1}
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208871, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.190@KRBTEST.COM for oryx/127.1.44.188@KRBTEST.COM
I20260924 00:14:32.707875 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7365
I20260924 00:14:32.709244 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.702630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38526)
0924 00:14:32.702675 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.702685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.702720 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.702847 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.702848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.702996 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.703030 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.703032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.703713 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.703714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.704215 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.704217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.704572 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.704573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.704633 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.705058 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.705063 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.706370 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.708147 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.708148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.708149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.708361 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.708572 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.708573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.708573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.708631 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.708915 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.708916 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.708965 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.709048 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.709147 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 00:14:32.709316 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.702683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.1.44.188:45671)
0924 00:14:32.702719 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.702721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.702729 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:32.702782 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.702783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.702792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.702835 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.703113 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.703635 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.704307 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.704503 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.704574 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.706457 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.706459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.706460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.706466 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.708061 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.708451 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.708451 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.708452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.708501 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.708698 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.708699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.708699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.708838 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.708902 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.709102 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.709184 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:32.709694 7303 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.44.188:45671: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b002ba0cbd174921bc293680923e5a71"
member_type: NON_VOTER
last_known_addr {
host: "127.1.44.188"
port: 45671
}
attrs {
promote: true
}
}
dest_uuid: "1fd90ec71d534cf096671d64b6337386"
cas_config_opid_index: 5
I20260924 00:14:32.709909 7303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } }
I20260924 00:14:32.710408 7861 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b002ba0cbd174921bc293680923e5a71 (127.1.44.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.711313 7861 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.44.188:45671
I20260924 00:14:32.711380 7861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.711442 7861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:32.711763 7996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:32.713024 7996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Refusing update from remote peer 1fd90ec71d534cf096671d64b6337386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 00:14:32.713266 7862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:32.713798 7954 master.cc:537] Master@127.1.44.188:45671 shutting down...
I20260924 00:14:32.714763 7996 raft_consensus.cc:858] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } }
W20260924 00:14:32.714825 7996 raft_consensus.cc:1565] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } }
I20260924 00:14:32.715301 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:44703
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:37919
--webserver_port=35489
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:14:32.715353 7996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.718302 8048 mvcc.cc:205] Tried to move back new op lower bound from 7332695538691727360 to 7332695538506588160. Current Snapshot: MvccSnapshot[applied={T|T < 7332695538517065728}]
I20260924 00:14:32.719365 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd90ec71d534cf096671d64b6337386. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.719455 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260924 00:14:32.719516 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.719573 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260924 00:14:32.719609 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } }
I20260924 00:14:32.719655 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260924 00:14:32.720228 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.710282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:52026)
0924 00:14:32.715897 (+ 5615us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.715910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.715986 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.716211 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.716211 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.716213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:32.716250 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.716251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.717193 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.717194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.718429 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.718432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.719960 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.719961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.720027 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.720052 (+ 25us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.720111 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.720151 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5598}
I20260924 00:14:32.720290 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.710367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.1.44.189:45337)
0924 00:14:32.710444 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.710446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.716043 (+ 5597us) server_negotiation.cc:407] Connection header received
0924 00:14:32.716073 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.716075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.716098 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.716153 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:32.716316 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.717077 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.719141 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.719775 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.719823 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.719900 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.720141 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.720190 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 00:14:32.720507 7910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 LEARNER]: Refusing update from remote peer 1fd90ec71d534cf096671d64b6337386: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 00:14:32.720717 7862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 00:14:32.721288 7910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b002ba0cbd174921bc293680923e5a71 (127.1.44.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.721895 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.721973 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:32.722501 7939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd90ec71d534cf096671d64b6337386. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
W20260924 00:14:32.722579 7290 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 -> Peer b002ba0cbd174921bc293680923e5a71 (127.1.44.188:45671): Couldn't send request to peer b002ba0cbd174921bc293680923e5a71. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 00:14:32.722592 7939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:14:32.722635 7939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: NON_VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: true } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.722695 7939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:14:32.722755 7954 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260924 00:14:32.722851 7954 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260924 00:14:32.722882 7954 tablet_replica.cc:330] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: stopping tablet replica
I20260924 00:14:32.729092 7862 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: attempting to promote NON_VOTER 6eaaaf9c3dbe440693c907ffec519dd3 to VOTER
I20260924 00:14:32.729290 7862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } }
I20260924 00:14:32.729826 7910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 LEARNER]: Refusing update from remote peer 1fd90ec71d534cf096671d64b6337386: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 00:14:32.729916 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.729807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38538)
0924 00:14:32.729851 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.729887 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:45671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 00:14:32.730108 8058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 00:14:32.730217 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.730150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38550)
0924 00:14:32.730178 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.730197 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:45671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 00:14:32.730494 7290 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 -> Peer b002ba0cbd174921bc293680923e5a71 (127.1.44.188:45671): Couldn't send request to peer b002ba0cbd174921bc293680923e5a71. Status: Network error: Client connection negotiation failed: client connection to 127.1.44.188:45671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:14:32.730774 7862 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.731748 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.731822 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:32.731879 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.731926 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:32.732162 7910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.733000 8057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.733091 8057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:14:32.739682 7954 master.cc:552] Master@127.1.44.188:45671 shutdown complete.
I20260924 00:14:32.739728 7954 master_runner.cc:305] Clearing existing system tablet
I20260924 00:14:32.739851 7954 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.739893 7954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.740597 7954 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:32.741155 8060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.741307 7954 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.741364 7954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.741387 7954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
uuid: "b002ba0cbd174921bc293680923e5a71"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "d8b1f1b475d5bf7502660f5a589a80d6"
server_key_iv: "1ff3aa3dfadac52b6e00a7c42a3acc38"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.742020 7954 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:32.743489 7954 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 00:14:32.743552 7954 log.cc:1196] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 00:14:32.743772 7954 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Deleting consensus metadata
I20260924 00:14:32.743881 7954 master_runner.cc:315] Copying system tablet from 127.1.44.190:37919
I20260924 00:14:32.744457 7954 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: tablet copy: Beginning tablet copy session from remote peer at address 127.1.44.190:37919
I20260924 00:14:32.751701 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.744807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.1.44.190:37919)
0924 00:14:32.744846 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.744848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.744922 (+ 74us) server_negotiation.cc:407] Connection header received
0924 00:14:32.744981 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.744982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.744992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.745056 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.745399 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.746146 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.747155 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.747478 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.747546 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.748637 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.748639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.748639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.748646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.750450 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.750853 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.750854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.750854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.750923 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.751144 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.751145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.751145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.751299 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.751369 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.751542 (+ 173us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.751616 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:32.751705 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.744753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45156)
0924 00:14:32.744864 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.744874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.744932 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.745059 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.745060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.745255 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.745319 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.745321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.746232 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.746233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.747068 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.747071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.747571 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.747572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.747641 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.748145 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.748151 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.748532 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.750532 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.750533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.750534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.750772 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.751000 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.751001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.751001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.751073 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.751375 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.751376 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.751424 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.751504 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.751614 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260924 00:14:32.752315 7323 tablet_copy_service.cc:140] P 1fd90ec71d534cf096671d64b6337386: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b002ba0cbd174921bc293680923e5a71 ({username='oryx', principal='oryx/127.1.44.188@KRBTEST.COM'} at 127.0.0.1:45156)
I20260924 00:14:32.752377 7323 tablet_copy_service.cc:161] P 1fd90ec71d534cf096671d64b6337386: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b002ba0cbd174921bc293680923e5a71 at {username='oryx', principal='oryx/127.1.44.188@KRBTEST.COM'} at 127.0.0.1:45156: session id = b002ba0cbd174921bc293680923e5a71-00000000000000000000000000000000
I20260924 00:14:32.752944 7323 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 00:14:32.753355 7954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 00:14:32.754483 7954 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: tablet copy: Starting download of 0 data blocks...
I20260924 00:14:32.754614 7954 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: tablet copy: Starting download of 1 WAL segments...
I20260924 00:14:32.755352 7954 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 00:14:32.756143 7323 tablet_copy_service.cc:342] P 1fd90ec71d534cf096671d64b6337386: Request end of tablet copy session b002ba0cbd174921bc293680923e5a71-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.44.188@KRBTEST.COM'} at 127.0.0.1:45156
I20260924 00:14:32.756196 7323 tablet_copy_service.cc:433] P 1fd90ec71d534cf096671d64b6337386: ending tablet copy session b002ba0cbd174921bc293680923e5a71-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b002ba0cbd174921bc293680923e5a71
I20260924 00:14:32.758325 7954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.759418 8070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.759498 8067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.759691 8068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.766088 7954 server_base.cc:1034] running on GCE node
I20260924 00:14:32.766223 7954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.766362 7954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.767519 7954 hybrid_clock.cc:648] HybridClock initialized: now 1790208872767495 us; error 45 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.769057 7954 init.cc:377] Logged in from keytab as oryx/127.1.44.188@KRBTEST.COM (short username oryx)
I20260924 00:14:32.769335 7954 webserver.cc:460] Webserver started at http://127.1.44.188:41497/ using document root <none> and password file <none>
I20260924 00:14:32.769452 7954 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.769486 7954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.770190 7954 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:32.770780 8077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.770955 7954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.771005 7954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
uuid: "b002ba0cbd174921bc293680923e5a71"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "d8b1f1b475d5bf7502660f5a589a80d6"
server_key_iv: "1ff3aa3dfadac52b6e00a7c42a3acc38"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.771063 7954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.781599 7954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.783505 7954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.786600 7954 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.188:45671
I20260924 00:14:32.786947 7954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
I20260924 00:14:32.787177 8129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.188:45671 every 8 connection(s)
I20260924 00:14:32.791978 8130 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:14:32.792429 8130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Bootstrap starting.
I20260924 00:14:32.794672 8130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:14:32.794880 8130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Bootstrap complete.
I20260924 00:14:32.795070 8130 raft_consensus.cc:348] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } }
I20260924 00:14:32.795161 8130 raft_consensus.cc:729] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b002ba0cbd174921bc293680923e5a71, State: Initialized, Role: LEARNER
I20260924 00:14:32.795219 8130 consensus_queue.cc:260] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } }
I20260924 00:14:32.795377 8130 sys_catalog.cc:564] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:32.796828 8144 catalog_manager.cc:1259] Loaded cluster ID: 7ff6785193474eb4b423f2110201643c
I20260924 00:14:32.796921 8144 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: loading cluster ID for follower catalog manager: success
I20260924 00:14:32.797031 8133 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:32.797125 8133 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:14:32.800354 8144 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: acquiring CA information for follower catalog manager: success
I20260924 00:14:32.800709 8144 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 00:14:32.816291 8051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:32.816489 8051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:32.816511 8051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:32.817648 8051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:32.817688 8051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:32.817713 8051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:32.819162 8051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:44703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=35489
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:32.819454 8051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:32.819700 8051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:14:32.824513 8051 server_base.cc:1034] running on GCE node
W20260924 00:14:32.824460 8149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.824456 8150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.824448 8152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.824946 8051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.825181 8051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.826345 8051 hybrid_clock.cc:648] HybridClock initialized: now 1790208872826342 us; error 36 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.828691 8051 init.cc:377] Logged in from keytab as oryx/127.1.44.129@KRBTEST.COM (short username oryx)
I20260924 00:14:32.829097 8051 webserver.cc:460] Webserver started at http://127.1.44.129:35489/ using document root <none> and password file <none>
I20260924 00:14:32.829317 8051 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.829396 8051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.830654 8051 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:32.831182 8159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.831363 8051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:32.831457 8051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc"
format_stamp: "Formatted at 2026-09-24 00:14:31 on dist-test-slave-cm7b"
server_key: "830acc9966fa1cbf1bb807747ae426c7"
server_key_iv: "60985ce05ce5f990d483476a35259547"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.831784 8051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.850513 8051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:32.852967 8051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:32.853116 8051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:32.853332 8051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:32.853633 8051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:32.853686 8051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.853727 8051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:32.853755 8051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.859679 8051 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:44703
I20260924 00:14:32.859786 8272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:44703 every 8 connection(s)
I20260924 00:14:32.860088 8051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:32.860955 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 8051
I20260924 00:14:32.861115 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7499
I20260924 00:14:32.867003 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:44951
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:37919
--webserver_port=42387
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.188@KRBTEST.COM
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:32.876267 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.861181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:33551 (local address 127.1.44.189:45337)
0924 00:14:32.861584 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.861586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.861596 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:32.863404 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.863405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.863419 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.863468 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.864712 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.865205 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.866808 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.867135 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.867227 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.870222 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.870224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.870225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.870232 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.874597 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.875086 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.875088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.875088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.875150 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.875475 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.875476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.875477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.875629 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.875674 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.876088 (+ 414us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.876165 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260924 00:14:32.876783 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.860787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.1.44.129:33551)
0924 00:14:32.860955 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.860996 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.863294 (+ 2298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.863549 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.863553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.864338 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.864613 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.864620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.865300 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.865301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.866707 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.866709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.867319 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.867321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.867477 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.868058 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.868067 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.870115 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.874681 (+ 4566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.874687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.874689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.874997 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.875246 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.875246 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.875247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.875388 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.875722 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.875728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.875904 (+ 176us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.876357 (+ 453us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.876529 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260924 00:14:32.877224 8275 heartbeater.cc:347] Connected to a master server at 127.1.44.189:45337
I20260924 00:14:32.877310 8275 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:32.877447 8275 heartbeater.cc:511] Master 127.1.44.189:45337 requested a full tablet report, sending...
I20260924 00:14:32.877935 7900 ts_manager.cc:194] Registered new tserver with Master: 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703)
I20260924 00:14:32.878744 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.861682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:48977 (local address 127.1.44.190:37919)
0924 00:14:32.861755 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.861757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.866981 (+ 5224us) server_negotiation.cc:407] Connection header received
0924 00:14:32.867015 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.867016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.867029 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.867087 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.867556 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.868306 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.871589 (+ 3283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.871988 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.872037 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.874766 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.874767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.874768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.874775 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.876510 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.877004 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.877005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.877006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.877063 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.878117 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.878118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.878118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.878255 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.878308 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.878588 (+ 280us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.878659 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 00:14:32.878749 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.861059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:48977)
0924 00:14:32.866902 (+ 5843us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.866912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.866968 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.867137 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.867138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.867376 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.867467 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.867469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.868408 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.868409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.869297 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.869301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.872124 (+ 2823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.872125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.872201 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.872659 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.872665 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.874674 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.876593 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.876593 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.876594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.876779 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.877136 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.877137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.877137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.878030 (+ 893us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.878443 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.878443 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.878475 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.878549 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.878648 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5827,"thread_start_us":65,"threads_started":1}
I20260924 00:14:32.879123 8274 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37919
I20260924 00:14:32.879166 8274 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:32.879267 8274 heartbeater.cc:511] Master 127.1.44.190:37919 requested a full tablet report, sending...
I20260924 00:14:32.879511 7303 ts_manager.cc:194] Re-registered known tserver with Master: 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703)
I20260924 00:14:32.880053 7303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:48977
I20260924 00:14:32.881593 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.860781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:56009 (local address 127.1.44.188:45671)
0924 00:14:32.867337 (+ 6556us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:32.867340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:32.867348 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:32.867371 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:32.867372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:32.867384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:32.868181 (+ 797us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:32.868224 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.869239 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.870356 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.870962 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.877623 (+ 6661us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.877674 (+ 51us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:32.877675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:32.877675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:32.877680 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:32.880408 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.880767 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.880768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.880768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.880825 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:32.881033 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:32.881033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:32.881034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:32.881213 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:32.881270 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:32.881422 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 00:14:32.881501 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6256,"thread_start_us":71,"threads_started":1}
I20260924 00:14:32.881771 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:32.860787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.1.44.129:56009)
0924 00:14:32.861938 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:32.861947 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:32.863311 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:32.867476 (+ 4165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:32.867477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:32.868055 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:32.868104 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.868106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.869377 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.869379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.870221 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:32.870223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:32.871335 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:32.871336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:32.871401 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:32.871902 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:32.871907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:32.874368 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:32.880493 (+ 6125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.880494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.880494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.880683 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.880891 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:32.880892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:32.880893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:32.880952 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:32.881272 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:32.881273 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:32.881311 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 00:14:32.881396 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 00:14:32.881498 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":4,"thread_start_us":79,"threads_started":1}
I20260924 00:14:32.882110 8273 heartbeater.cc:347] Connected to a master server at 127.1.44.188:45671
I20260924 00:14:32.882172 8273 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:32.882242 8273 heartbeater.cc:511] Master 127.1.44.188:45671 requested a full tablet report, sending...
I20260924 00:14:32.882581 8094 ts_manager.cc:194] Registered new tserver with Master: 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703)
W20260924 00:14:32.952028 8281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:32.952214 8281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:32.952236 8281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:32.953296 8281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:32.953330 8281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:32.953356 8281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:32.954712 8281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:44951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=42387
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:32.954891 8281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:32.955066 8281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:32.960002 8286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:32.960011 8289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.960073 8281 server_base.cc:1034] running on GCE node
W20260924 00:14:32.959986 8287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:32.960381 8281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:32.960618 8281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:32.961793 8281 hybrid_clock.cc:648] HybridClock initialized: now 1790208872961772 us; error 29 us; skew 500 ppm
Sep 24 00:14:32 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:32.964260 8281 init.cc:377] Logged in from keytab as oryx/127.1.44.130@KRBTEST.COM (short username oryx)
I20260924 00:14:32.964721 8281 webserver.cc:460] Webserver started at http://127.1.44.130:42387/ using document root <none> and password file <none>
I20260924 00:14:32.964954 8281 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:32.965034 8281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:32.966264 8281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:32.966832 8296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:32.966993 8281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:32.967067 8281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "f61e8ac142799e099437e5fa550a1760"
server_key_iv: "e7be1f1d1d4e71b27cbec8641d14196b"
server_key_version: "encryptionkey@0"
I20260924 00:14:32.967504 8281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:32.999794 8281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:33.001833 8281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:33.002003 8281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:33.002274 8281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:33.002596 8281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:33.002651 8281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:33.002693 8281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:33.002722 8281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:33.009277 8281 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:44951
I20260924 00:14:33.009320 8409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:44951 every 8 connection(s)
I20260924 00:14:33.009678 8281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:33.012548 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 8281
I20260924 00:14:33.012717 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7633
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.188@KRBTEST.COM
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208872, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:33.020440 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:44661
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:37919
--webserver_port=43921
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:14:33.023181 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.010742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:44405 (local address 127.1.44.189:45337)
0924 00:14:33.010803 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.010805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.010815 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:33.012122 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.012127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.012137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.012202 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.013347 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.014216 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.015487 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.015844 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.015895 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.019753 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.019755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.019755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.019762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.021492 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.022022 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.022024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.022024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.022088 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.022401 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.022403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.022403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.022565 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.022620 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.022991 (+ 371us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.023074 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:14:33.023779 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.010412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.1.44.130:44405)
0924 00:14:33.010604 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.010633 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.011879 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.012222 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.012223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.012572 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:33.013251 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.013253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.014314 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.014315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.015383 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.015386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.016013 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.016014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.016173 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.017246 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:33.017251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:33.019651 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:33.021587 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.021593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.021595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.021935 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.022160 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.022161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.022162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.022316 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.022655 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:33.022661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:33.022828 (+ 167us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.023302 (+ 474us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.023498 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":33,"thread_start_us":131,"threads_started":1}
I20260924 00:14:33.024250 8412 heartbeater.cc:347] Connected to a master server at 127.1.44.189:45337
I20260924 00:14:33.024381 8412 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:33.024519 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.010431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:55237 (local address 127.1.44.190:37919)
0924 00:14:33.010505 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.010508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.010709 (+ 201us) server_negotiation.cc:407] Connection header received
0924 00:14:33.011991 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.011993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.012005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.012059 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.014372 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.015155 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.016173 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.016522 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.016561 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.021262 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.021265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.021265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.021272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.022939 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.023543 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.023545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.023545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.023602 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.023790 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.023791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.023792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.023919 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.024018 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.024317 (+ 299us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.024417 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:33.024586 8412 heartbeater.cc:511] Master 127.1.44.189:45337 requested a full tablet report, sending...
I20260924 00:14:33.024892 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.010411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:55237)
0924 00:14:33.010590 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.010632 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.011876 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.012140 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.012143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.012577 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:33.014269 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.014271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.015285 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.015286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.016071 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.016072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.016638 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.016639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.016699 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.017102 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:33.017107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:33.021168 (+ 4061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:33.023026 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.023027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.023028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.023244 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.023675 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.023676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.023676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.023728 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.024010 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:33.024011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:33.024054 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.024121 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.024214 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1476,"thread_start_us":112,"threads_started":1}
I20260924 00:14:33.024981 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.010874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:39207 (local address 127.1.44.188:45671)
0924 00:14:33.010931 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.010933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.011391 (+ 458us) server_negotiation.cc:407] Connection header received
0924 00:14:33.012259 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.012260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.012272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.012327 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.012993 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.013776 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.015477 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.015846 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.015905 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.021114 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.021115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.021116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.021124 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.023374 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.023776 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.023777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.023777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.023834 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.024398 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.024399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.024399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.024528 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.024579 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.024761 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.024855 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:14:33.025064 7900 ts_manager.cc:194] Registered new tserver with Master: 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951)
I20260924 00:14:33.025197 8411 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37919
I20260924 00:14:33.025243 8411 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:33.025318 8411 heartbeater.cc:511] Master 127.1.44.190:37919 requested a full tablet report, sending...
I20260924 00:14:33.025594 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.010718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.1.44.130:39207)
0924 00:14:33.011303 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.011313 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.011885 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.012349 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.012349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.012585 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:33.012886 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.012899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.013928 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.013929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.015381 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.015383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.016016 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.016017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.016173 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.017032 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:33.017046 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:33.021001 (+ 3955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:33.023467 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.023468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.023469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.023690 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.024253 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.024254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.024254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.024317 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.024613 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:33.024614 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:33.024649 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.025079 (+ 430us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.025190 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":14,"thread_start_us":63,"threads_started":1}
I20260924 00:14:33.025701 7303 ts_manager.cc:194] Re-registered known tserver with Master: 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951)
I20260924 00:14:33.025926 8410 heartbeater.cc:347] Connected to a master server at 127.1.44.188:45671
I20260924 00:14:33.025976 8410 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:33.026041 8410 heartbeater.cc:511] Master 127.1.44.188:45671 requested a full tablet report, sending...
I20260924 00:14:33.026094 7303 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:55237
I20260924 00:14:33.026319 8094 ts_manager.cc:194] Registered new tserver with Master: 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951)
W20260924 00:14:33.106051 8417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:33.106245 8417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:33.106268 8417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:33.107651 8417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:33.107692 8417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:33.107718 8417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:33.109167 8417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:44661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=43921
--tserver_master_addrs=127.1.44.190:37919,127.1.44.189:45337,127.1.44.188:45671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:33.109400 8417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:33.109648 8417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:33.114539 8423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:33.114549 8425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:33.114570 8422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:33.115072 8417 server_base.cc:1034] running on GCE node
I20260924 00:14:33.115341 8417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:33.115607 8417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:33.116791 8417 hybrid_clock.cc:648] HybridClock initialized: now 1790208873116778 us; error 29 us; skew 500 ppm
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208873, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:33.119944 8417 init.cc:377] Logged in from keytab as oryx/127.1.44.131@KRBTEST.COM (short username oryx)
I20260924 00:14:33.120612 8417 webserver.cc:460] Webserver started at http://127.1.44.131:43921/ using document root <none> and password file <none>
I20260924 00:14:33.120888 8417 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:33.120968 8417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:33.122855 8417 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:14:33.123572 8432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:33.123729 8417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:33.123829 8417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "161e7e7cf01244ea85e124b73c88d69c"
format_stamp: "Formatted at 2026-09-24 00:14:32 on dist-test-slave-cm7b"
server_key: "db4790588fb4c3f81b493128a49e1d08"
server_key_iv: "9d9e62b1e3433ec75feb0aae26f7a1f9"
server_key_version: "encryptionkey@0"
I20260924 00:14:33.124171 8417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:33.143872 8417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:33.145880 8417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:33.146035 8417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:33.146297 8417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:33.146608 8417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:33.146641 8417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:33.146663 8417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:33.146709 8417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:33.152611 8417 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:44661
I20260924 00:14:33.152652 8545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:44661 every 8 connection(s)
I20260924 00:14:33.152971 8417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:33.158305 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 8417
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208873, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208873, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.189@KRBTEST.COM
Sep 24 00:14:33 dist-test-slave-cm7b krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208873, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.188@KRBTEST.COM
I20260924 00:14:33.166237 7951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.154091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:41153 (local address 127.1.44.189:45337)
0924 00:14:33.154255 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.154257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.154268 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:14:33.154748 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.154749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.154761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.154794 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.155651 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.156144 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.157564 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.157872 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.157944 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.162138 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.162140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.162140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.162147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.164540 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.165046 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.165047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.165048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.165106 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.165383 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.165384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.165385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.165536 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.165609 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.166045 (+ 436us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.166109 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260924 00:14:33.166468 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.153819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.1.44.131:41153)
0924 00:14:33.153981 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.154030 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.154629 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.154872 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.154876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.155330 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:33.155547 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.155552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.156246 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.156247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.157466 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.157468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.157948 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.157949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.158092 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.158796 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:33.158800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:33.162035 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:33.164636 (+ 2601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.164642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.164644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.164958 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.165178 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.165179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.165179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.165307 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.165714 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:33.165720 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:33.165898 (+ 178us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.166025 (+ 127us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.166195 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":21,"thread_start_us":88,"threads_started":1}
I20260924 00:14:33.166883 8548 heartbeater.cc:347] Connected to a master server at 127.1.44.189:45337
I20260924 00:14:33.166980 8548 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:33.167184 8548 heartbeater.cc:511] Master 127.1.44.189:45337 requested a full tablet report, sending...
I20260924 00:14:33.167644 7900 ts_manager.cc:194] Registered new tserver with Master: 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661)
I20260924 00:14:33.169361 8557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.159350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.1.44.190:37919)
0924 00:14:33.159882 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.159884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.159893 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:33.159956 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.159957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.159971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.160028 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.160810 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.161635 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.162663 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.163004 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.163075 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.164720 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.164722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.164722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.164729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.167809 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.168235 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.168236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.168236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.168294 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.168507 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.168508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.168508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.168654 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.168747 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.169146 (+ 399us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.169224 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":76,"threads_started":1}
I20260924 00:14:33.169838 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.154371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:50269 (local address 127.1.44.188:45671)
0924 00:14:33.154422 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.154424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.154432 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:33.154728 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.154729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.154740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.154784 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.155658 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.156434 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.157689 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.157985 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.158085 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.166798 (+ 8713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.166800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.166800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.166807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.168573 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.169039 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.169040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.169041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.169098 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.169296 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.169297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.169297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.169420 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.169464 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.169656 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.169731 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 00:14:33.169874 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.154107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.1.44.131:50269)
0924 00:14:33.154279 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.154290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.154636 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.155130 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.155131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.155446 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:33.155548 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.155554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.156532 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.156533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.157610 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.157612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.158086 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.158087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.158155 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.158735 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:33.158745 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:33.165640 (+ 6895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:33.168650 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.168651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.168651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.168923 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.169173 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.169174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.169174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.169229 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.169508 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:33.169509 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:33.169548 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.169619 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.169720 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":60,"thread_start_us":59,"threads_started":1}
I20260924 00:14:33.170181 8546 heartbeater.cc:347] Connected to a master server at 127.1.44.188:45671
I20260924 00:14:33.170228 8546 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:33.170295 8546 heartbeater.cc:511] Master 127.1.44.188:45671 requested a full tablet report, sending...
I20260924 00:14:33.170756 8094 ts_manager.cc:194] Registered new tserver with Master: 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661)
I20260924 00:14:33.171765 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.159606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.1.44.188:45671)
0924 00:14:33.159950 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.159952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.160065 (+ 113us) server_negotiation.cc:407] Connection header received
0924 00:14:33.160307 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.160308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.160320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.160354 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.161143 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.161886 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.162914 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.163238 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.163277 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.164802 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.164803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.164803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.164810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.166493 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.170575 (+ 4082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.170576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.170576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.170636 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.170994 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.170994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.170995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.171122 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.171190 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.171609 (+ 419us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.171687 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":65,"threads_started":1}
I20260924 00:14:33.172304 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.153819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:50309)
0924 00:14:33.154258 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.154267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.154635 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.154872 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.154876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.155439 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:33.155548 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.155554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.156533 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.156533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.157773 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.157775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.158263 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.158264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.158787 (+ 523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.159231 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:33.159236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:33.168087 (+ 8851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:33.170059 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.170060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.170060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.171498 (+ 1438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.171677 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:33.171678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:33.171679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:33.171741 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:33.172036 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:33.172037 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:33.172076 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.172149 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.172240 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":26,"thread_start_us":72,"threads_started":1}
I20260924 00:14:33.172354 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.154147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:50309 (local address 127.1.44.190:37919)
0924 00:14:33.154310 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.154312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.154320 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:33.154748 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.154749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.154761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.154794 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.155659 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.156435 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.157854 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.158144 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.158208 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.168188 (+ 9980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.168190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.168191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.168198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.169882 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.171552 (+ 1670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.171552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.171553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.171598 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.171807 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.171808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.171809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.171970 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.172037 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.172192 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.172268 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260924 00:14:33.171147 7303 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:33.173229 7303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:33.169380 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.159583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.1.44.189:45337)
0924 00:14:33.159821 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.159824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.160154 (+ 330us) server_negotiation.cc:407] Connection header received
0924 00:14:33.160453 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.160454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.160467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.160623 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.161350 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.162113 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.163057 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.163380 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.163416 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.164885 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.164886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.164886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.164893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.167642 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.168316 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.168317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.168317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.168371 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.168656 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.168657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.168657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.168843 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.168918 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.169177 (+ 259us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.169264 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":45,"threads_started":1}
I20260924 00:14:33.174149 8547 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37919
I20260924 00:14:33.174212 8547 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:33.174281 8547 heartbeater.cc:511] Master 127.1.44.190:37919 requested a full tablet report, sending...
I20260924 00:14:33.174991 7302 ts_manager.cc:194] Re-registered known tserver with Master: 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661)
I20260924 00:14:33.175498 7302 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:50309
I20260924 00:14:33.181550 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.178462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49002 (local address 127.1.44.129:44703)
0924 00:14:33.178616 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.178621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.178630 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:33.178673 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.178674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.178686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.178781 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:33.178898 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.179640 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.180714 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.181198 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.181296 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.181385 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.181437 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.181473 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260924 00:14:33.181632 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.177530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:44661 (local address 127.0.0.1:59372)
0924 00:14:33.177575 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.177585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.177619 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.178117 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.178118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.178120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:33.178157 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.178159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.179430 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.179431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.180549 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.180551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.181406 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.181407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.181465 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.181469 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.181497 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.181546 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 00:14:33.181540 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.177579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:44703 (local address 127.0.0.1:49002)
0924 00:14:33.177715 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.177725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.177767 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.178779 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.178780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.178782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:33.178837 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.178839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.179747 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.179748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.180633 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.180635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.181301 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.181302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.181359 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.181362 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.181420 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.181459 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260924 00:14:33.181941 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.177595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.1.44.131:44661)
0924 00:14:33.177829 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.177834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.177844 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:33.177933 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.177936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.177954 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.178120 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:33.178238 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.179338 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.180643 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.181315 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.181587 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.181659 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.181683 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.181860 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260924 00:14:33.182057 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.177757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44951 (local address 127.0.0.1:34554)
0924 00:14:33.178196 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.178206 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.178246 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.178528 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.178529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.178532 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:33.178566 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.178568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.179809 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.179811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.180932 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.180934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.181835 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.181836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.181894 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.181898 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.181926 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.181968 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":100,"threads_started":1}
I20260924 00:14:33.182415 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.177865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.1.44.130:44951)
0924 00:14:33.178357 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.178362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.178370 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:33.178406 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.178407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.178431 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.178630 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:33.178685 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.179717 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.181020 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.181747 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.182209 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.182281 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.182305 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.182344 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":68,"threads_started":1}
I20260924 00:14:33.182586 8480 tablet_service.cc:1467] Processing CreateTablet for tablet 507c7fe43a314cc5a69837b5e147e028 (DEFAULT_TABLE table=test-table [id=4772eb2969e34e3dbfa84806ffccb1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:33.182922 8480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507c7fe43a314cc5a69837b5e147e028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:33.182964 8344 tablet_service.cc:1467] Processing CreateTablet for tablet 507c7fe43a314cc5a69837b5e147e028 (DEFAULT_TABLE table=test-table [id=4772eb2969e34e3dbfa84806ffccb1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:33.183058 8207 tablet_service.cc:1467] Processing CreateTablet for tablet 507c7fe43a314cc5a69837b5e147e028 (DEFAULT_TABLE table=test-table [id=4772eb2969e34e3dbfa84806ffccb1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:33.183326 8344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507c7fe43a314cc5a69837b5e147e028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:33.183437 8207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 507c7fe43a314cc5a69837b5e147e028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:33.187310 8574 tablet_bootstrap.cc:492] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Bootstrap starting.
I20260924 00:14:33.187850 8574 tablet_bootstrap.cc:654] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:33.188144 8574 log.cc:824] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:33.188884 8574 tablet_bootstrap.cc:492] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: No bootstrap required, opened a new log
I20260924 00:14:33.188935 8574 ts_tablet_manager.cc:1397] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:33.189836 8573 tablet_bootstrap.cc:492] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Bootstrap starting.
I20260924 00:14:33.190699 8573 tablet_bootstrap.cc:654] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:33.190981 8573 log.cc:824] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:33.191042 8574 raft_consensus.cc:348] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.191267 8574 raft_consensus.cc:374] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:33.191314 8574 raft_consensus.cc:729] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3d5ce1fbd4439db16d7c2123d1ebbc, State: Initialized, Role: FOLLOWER
I20260924 00:14:33.191339 8572 tablet_bootstrap.cc:492] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Bootstrap starting.
I20260924 00:14:33.191431 8574 consensus_queue.cc:260] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.191722 8274 heartbeater.cc:503] Master 127.1.44.190:37919 was elected leader, sending a full tablet report...
I20260924 00:14:33.191735 8573 tablet_bootstrap.cc:492] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: No bootstrap required, opened a new log
I20260924 00:14:33.191797 8573 ts_tablet_manager.cc:1397] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:33.191906 8574 ts_tablet_manager.cc:1428] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:33.192071 8572 tablet_bootstrap.cc:654] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:33.192340 8572 log.cc:824] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:33.193951 8572 tablet_bootstrap.cc:492] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: No bootstrap required, opened a new log
I20260924 00:14:33.194041 8572 ts_tablet_manager.cc:1397] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260924 00:14:33.194873 8573 raft_consensus.cc:348] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.195034 8573 raft_consensus.cc:374] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:33.195068 8573 raft_consensus.cc:729] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1138b3d8b4bf4acba32cc3fb98079fb4, State: Initialized, Role: FOLLOWER
I20260924 00:14:33.195235 8573 consensus_queue.cc:260] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.195500 8573 ts_tablet_manager.cc:1428] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 00:14:33.195529 8411 heartbeater.cc:503] Master 127.1.44.190:37919 was elected leader, sending a full tablet report...
I20260924 00:14:33.196143 8572 raft_consensus.cc:348] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.196295 8572 raft_consensus.cc:374] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:33.196337 8572 raft_consensus.cc:729] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161e7e7cf01244ea85e124b73c88d69c, State: Initialized, Role: FOLLOWER
I20260924 00:14:33.196436 8572 consensus_queue.cc:260] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.196689 8572 ts_tablet_manager.cc:1428] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 00:14:33.196738 8547 heartbeater.cc:503] Master 127.1.44.190:37919 was elected leader, sending a full tablet report...
I20260924 00:14:33.260512 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.257795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38562)
0924 00:14:33.257868 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.257884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.257931 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.258104 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.258104 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.258106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:33.258151 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.258153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.258990 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.258991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.259797 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.259800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.260327 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.260328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.260375 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.260378 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.260411 (+ 33us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.260452 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
W20260924 00:14:33.260514 8413 tablet.cc:2369] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:33.260551 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.257905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.1.44.188:45671)
0924 00:14:33.257971 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.257973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.257981 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:33.258006 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.258007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.258028 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.258086 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:33.258210 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.258807 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.259871 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.260253 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.260310 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.260396 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.260427 (+ 31us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.260463 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 00:14:33.260903 8104 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260924 00:14:33.261353 8133 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd90ec71d534cf096671d64b6337386. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:33.261467 8133 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: LEARNER
I20260924 00:14:33.285156 8580 raft_consensus.cc:482] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:33.285257 8580 raft_consensus.cc:504] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.285554 8580 leader_election.cc:288] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703), 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951)
I20260924 00:14:33.288425 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.285819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:52587 (local address 127.1.44.129:44703)
0924 00:14:33.285894 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.285896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.285915 (+ 19us) server_negotiation.cc:407] Connection header received
0924 00:14:33.285982 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.285983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.285997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.286055 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:33.286217 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.286678 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.287706 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.288069 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.288157 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.288166 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.288319 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.288355 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:33.288472 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.285743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:44703 (local address 127.1.44.131:52587)
0924 00:14:33.285819 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.285831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.285895 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.286068 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.286069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.286070 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:33.286138 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.286140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.286760 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.286761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.287614 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.287616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.288161 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.288162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.288242 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.288246 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.288313 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.288351 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 00:14:33.288486 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.286010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:51085 (local address 127.1.44.130:44951)
0924 00:14:33.286083 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.286085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.286092 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:33.286121 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.286122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.286132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.286193 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:33.286340 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.286797 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.287723 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.288128 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.288216 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.288224 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.288375 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.288412 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 00:14:33.288650 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.285889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44951 (local address 127.1.44.131:51085)
0924 00:14:33.285941 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:33.285950 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:33.285983 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:33.286196 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:33.286197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:33.286198 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:33.286268 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.286270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.286879 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.286880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.287636 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:33.287638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.288213 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.288214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:33.288292 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.288295 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:33.288358 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 00:14:33.288398 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 00:14:33.288836 8227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507c7fe43a314cc5a69837b5e147e028" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" is_pre_election: true
I20260924 00:14:33.289016 8227 raft_consensus.cc:2454] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 161e7e7cf01244ea85e124b73c88d69c in term 0.
I20260924 00:14:33.289311 8436 leader_election.cc:302] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 161e7e7cf01244ea85e124b73c88d69c, 2a3d5ce1fbd4439db16d7c2123d1ebbc; no voters:
I20260924 00:14:33.289412 8364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507c7fe43a314cc5a69837b5e147e028" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" is_pre_election: true
I20260924 00:14:33.289489 8580 raft_consensus.cc:2781] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:33.289546 8580 raft_consensus.cc:482] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:33.289561 8364 raft_consensus.cc:2454] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 161e7e7cf01244ea85e124b73c88d69c in term 0.
I20260924 00:14:33.289579 8580 raft_consensus.cc:3037] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:33.290299 8580 raft_consensus.cc:504] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.290442 8580 leader_election.cc:288] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 election: Requested vote from peers 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703), 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951)
I20260924 00:14:33.290593 8364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507c7fe43a314cc5a69837b5e147e028" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
I20260924 00:14:33.290648 8364 raft_consensus.cc:3037] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:33.290606 8227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "507c7fe43a314cc5a69837b5e147e028" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc"
I20260924 00:14:33.290700 8227 raft_consensus.cc:3037] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:33.291330 8364 raft_consensus.cc:2454] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 161e7e7cf01244ea85e124b73c88d69c in term 1.
I20260924 00:14:33.291510 8436 leader_election.cc:302] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1138b3d8b4bf4acba32cc3fb98079fb4, 161e7e7cf01244ea85e124b73c88d69c; no voters:
I20260924 00:14:33.291610 8580 raft_consensus.cc:2781] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:33.291705 8227 raft_consensus.cc:2454] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 161e7e7cf01244ea85e124b73c88d69c in term 1.
I20260924 00:14:33.291774 8580 raft_consensus.cc:686] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 1 LEADER]: Becoming Leader. State: Replica: 161e7e7cf01244ea85e124b73c88d69c, State: Running, Role: LEADER
I20260924 00:14:33.291872 8580 consensus_queue.cc:237] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } }
I20260924 00:14:33.292912 7303 catalog_manager.cc:5495] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c reported cstate change: term changed from 0 to 1, leader changed from <none> to 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131). New cstate: current_term: 1 leader_uuid: "161e7e7cf01244ea85e124b73c88d69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:33.293509 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: NON_VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: true } } }
I20260924 00:14:33.293637 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:33.294220 7862 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: attempting to promote NON_VOTER b002ba0cbd174921bc293680923e5a71 to VOTER
I20260924 00:14:33.294338 7862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:33.294876 8104 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 LEARNER]: Refusing update from remote peer 1fd90ec71d534cf096671d64b6337386: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 00:14:33.294915 7910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Refusing update from remote peer 1fd90ec71d534cf096671d64b6337386: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 00:14:33.295121 8586 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260924 00:14:33.295274 8058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 00:14:33.296007 7862 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b002ba0cbd174921bc293680923e5a71 (127.1.44.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.296175 7910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b002ba0cbd174921bc293680923e5a71 (127.1.44.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.297012 8104 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b002ba0cbd174921bc293680923e5a71 (127.1.44.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
W20260924 00:14:33.367142 8276 tablet.cc:2369] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:33.393261 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.393419 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:33.393366 7861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.393476 7861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:33.393481 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.393627 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:33.393703 8588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.393756 8133 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fd90ec71d534cf096671d64b6337386" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:33.393814 8588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:14:33.393852 8133 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:14:33.398779 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.395500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.1.44.131:44661)
0924 00:14:33.395562 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.395564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.395617 (+ 53us) server_negotiation.cc:407] Connection header received
0924 00:14:33.395658 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.395659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.395681 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.395770 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:33.395932 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.396508 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.397399 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.397624 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.397725 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.397885 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:33.398003 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:33.398095 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.398117 (+ 22us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.398139 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:14:33.400617 8227 raft_consensus.cc:1261] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Refusing update from remote peer 161e7e7cf01244ea85e124b73c88d69c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:33.400663 8364 raft_consensus.cc:1261] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Refusing update from remote peer 161e7e7cf01244ea85e124b73c88d69c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:33.400956 8584 consensus_queue.cc:1035] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:33.401093 8580 consensus_queue.cc:1035] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:33.405491 7303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:33.405586 7303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:33.405922 8592 mvcc.cc:205] Tried to move back new op lower bound from 7332695545446350848 to 7332695545004179456. Current Snapshot: MvccSnapshot[applied={T|T < 7332695545446350848}]
I20260924 00:14:33.407238 7303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fd90ec71d534cf096671d64b6337386: Sending DeleteTablet for 3 replicas of tablet 507c7fe43a314cc5a69837b5e147e028
I20260924 00:14:33.407583 8480 tablet_service.cc:1514] Processing DeleteTablet for tablet 507c7fe43a314cc5a69837b5e147e028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:33 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:59372
I20260924 00:14:33.408056 8207 tablet_service.cc:1514] Processing DeleteTablet for tablet 507c7fe43a314cc5a69837b5e147e028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:33 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:49002
I20260924 00:14:33.408190 8344 tablet_service.cc:1514] Processing DeleteTablet for tablet 507c7fe43a314cc5a69837b5e147e028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:33 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:34554
I20260924 00:14:33.408710 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7271
I20260924 00:14:33.408720 8601 tablet_replica.cc:330] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: stopping tablet replica
I20260924 00:14:33.408825 8601 raft_consensus.cc:2229] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:33.409953 8601 raft_consensus.cc:2258] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:33.410424 8601 ts_tablet_manager.cc:1905] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:33.411533 8599 tablet_replica.cc:330] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: stopping tablet replica
I20260924 00:14:33.411625 8599 raft_consensus.cc:2229] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:33.411710 8599 raft_consensus.cc:2258] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:33.412127 8599 ts_tablet_manager.cc:1905] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:33.413203 8602 tablet_replica.cc:330] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: stopping tablet replica
I20260924 00:14:33.413298 8602 raft_consensus.cc:2229] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:33.413391 8602 raft_consensus.cc:2258] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:33.413811 8602 ts_tablet_manager.cc:1905] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:33.418221 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.408203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.1.44.188:45671)
0924 00:14:33.408241 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.408243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.409844 (+ 1601us) server_negotiation.cc:407] Connection header received
0924 00:14:33.409878 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.409879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.409891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.409952 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.410738 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.411523 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.412565 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.412870 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.412914 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.415049 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.415051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.415051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.415059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.416836 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.417254 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.417256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.417256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.417317 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.417581 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.417583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.417583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.417722 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.417798 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.418015 (+ 217us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.418101 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:33.424782 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.417301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52038 (local address 127.1.44.189:45337)
0924 00:14:33.417341 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.417343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.417579 (+ 236us) server_negotiation.cc:407] Connection header received
0924 00:14:33.417676 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.417677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.417691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.417720 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.418684 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.419164 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.420080 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.420299 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.420385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.421798 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.421799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.421799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.421807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.423536 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.423937 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.423939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.423939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.424004 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.424226 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.424227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.424227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.424379 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.424473 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.424603 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.424686 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:33.425554 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:33.417289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.1.44.188:45671)
0924 00:14:33.417329 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:33.417331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:33.417637 (+ 306us) server_negotiation.cc:407] Connection header received
0924 00:14:33.417728 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:33.417729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:33.417739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:33.417766 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:33.418456 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.419228 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.420389 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:33.420682 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:33.420720 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:33.422079 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:33.422080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:33.422081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:33.422087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:33.424304 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.424667 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.424668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.424668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.424720 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:33.425031 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:33.425032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:33.425032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:33.425148 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:33.425213 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:33.425383 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 00:14:33.425451 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:33.491276 8601 ts_tablet_manager.cc:1918] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:33.491375 8601 log.cc:1196] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/507c7fe43a314cc5a69837b5e147e028
I20260924 00:14:33.491645 8601 ts_tablet_manager.cc:1939] T 507c7fe43a314cc5a69837b5e147e028 P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Deleting consensus metadata
I20260924 00:14:33.494233 8599 ts_tablet_manager.cc:1918] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:33.494323 8599 log.cc:1196] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/507c7fe43a314cc5a69837b5e147e028
I20260924 00:14:33.494623 8599 ts_tablet_manager.cc:1939] T 507c7fe43a314cc5a69837b5e147e028 P 161e7e7cf01244ea85e124b73c88d69c: Deleting consensus metadata
I20260924 00:14:33.495905 8602 ts_tablet_manager.cc:1918] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:33.495995 8602 log.cc:1196] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/507c7fe43a314cc5a69837b5e147e028
W20260924 00:14:33.496163 8162 connection.cc:441] server connection from 127.0.0.1:49002 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49002 (request call id 1) could send its response
W20260924 00:14:33.496163 8434 connection.cc:441] server connection from 127.0.0.1:59372 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59372 (request call id 1) could send its response
I20260924 00:14:33.496250 8602 ts_tablet_manager.cc:1939] T 507c7fe43a314cc5a69837b5e147e028 P 1138b3d8b4bf4acba32cc3fb98079fb4: Deleting consensus metadata
W20260924 00:14:33.530076 8300 connection.cc:441] server connection from 127.0.0.1:34554 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34554 (request call id 1) could send its response
I20260924 00:14:34.405428 8610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.405107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:59117)
0924 00:14:34.405248 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.405345 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
W20260924 00:14:34.405640 8547 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:37919 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:34.408077 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.407808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:36295)
0924 00:14:34.407943 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.408024 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
W20260924 00:14:34.408283 8411 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:37919 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:34.409008 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.408842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:36367)
0924 00:14:34.408929 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.408975 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":43,"threads_started":1}
W20260924 00:14:34.409225 8274 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:37919 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:34.908155 8613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fd90ec71d534cf096671d64b6337386)
I20260924 00:14:34.908274 8613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:34.908571 8614 raft_consensus.cc:482] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fd90ec71d534cf096671d64b6337386)
I20260924 00:14:34.908591 8613 leader_election.cc:288] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919), b002ba0cbd174921bc293680923e5a71 (127.1.44.188:45671)
I20260924 00:14:34.908705 8614 raft_consensus.cc:504] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:34.908967 8614 leader_election.cc:288] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919), 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189:45337)
I20260924 00:14:34.909294 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.908682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45650)
0924 00:14:34.908939 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.909246 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":46,"threads_started":1}
I20260924 00:14:34.909297 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.909019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45664)
0924 00:14:34.909109 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.909255 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
W20260924 00:14:34.909402 8081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
W20260924 00:14:34.909544 7887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:34.909788 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.909711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45680)
0924 00:14:34.909742 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.909764 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260924 00:14:34.909860 8081 leader_election.cc:334] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:34.910212 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.910106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45690)
0924 00:14:34.910159 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.910189 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260924 00:14:34.910285 7887 leader_election.cc:334] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:34.912099 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.908847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:45671 (local address 127.0.0.1:38580)
0924 00:14:34.909101 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.909112 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:34.909191 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:34.909387 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:34.909388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:34.909390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:34.909446 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:34.909449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.910426 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.910427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:34.911276 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:34.911279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.911834 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.911835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:34.911896 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:34.911900 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:34.911990 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 00:14:34.912030 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":35,"threads_started":1}
I20260924 00:14:34.912101 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.908963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38580 (local address 127.1.44.188:45671)
0924 00:14:34.909062 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:34.909065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:34.909162 (+ 97us) server_negotiation.cc:407] Connection header received
0924 00:14:34.909267 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:34.909267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:34.909278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:34.909315 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:34.909544 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.910362 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.911369 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.911742 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.911833 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:34.911844 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:34.911997 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 00:14:34.912045 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":80,"thread_start_us":43,"threads_started":1}
I20260924 00:14:34.912189 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.909084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45337 (local address 127.0.0.1:52044)
0924 00:14:34.909257 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:34.909266 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:34.909341 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:34.909546 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:34.909546 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:34.909548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:34.909620 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:34.909623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.910525 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.910526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:34.911346 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:34.911348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.911894 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.911895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:34.911961 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:34.911967 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 00:14:34.912045 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 00:14:34.912086 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":2,"thread_start_us":44,"threads_started":1}
I20260924 00:14:34.912184 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:34.909136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.1.44.189:45337)
0924 00:14:34.909210 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:34.909212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:34.909316 (+ 104us) server_negotiation.cc:407] Connection header received
0924 00:14:34.909427 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:34.909428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:34.909437 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:34.909484 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:34.909704 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.910424 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.911433 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:34.911800 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:34.911892 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:34.911903 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:34.912051 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 00:14:34.912088 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58,"thread_start_us":34,"threads_started":1}
I20260924 00:14:34.912653 8104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b002ba0cbd174921bc293680923e5a71" is_pre_election: true
I20260924 00:14:34.912684 7910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b002ba0cbd174921bc293680923e5a71" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" is_pre_election: true
I20260924 00:14:34.912750 8104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eaaaf9c3dbe440693c907ffec519dd3 in term 1.
I20260924 00:14:34.912770 7910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b002ba0cbd174921bc293680923e5a71 in term 1.
I20260924 00:14:34.912883 7887 leader_election.cc:302] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6eaaaf9c3dbe440693c907ffec519dd3, b002ba0cbd174921bc293680923e5a71; no voters: 1fd90ec71d534cf096671d64b6337386
I20260924 00:14:34.912911 8078 leader_election.cc:302] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6eaaaf9c3dbe440693c907ffec519dd3, b002ba0cbd174921bc293680923e5a71; no voters: 1fd90ec71d534cf096671d64b6337386
I20260924 00:14:34.912988 8613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 00:14:34.913005 8614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 00:14:34.913030 8613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fd90ec71d534cf096671d64b6337386)
I20260924 00:14:34.913030 8614 raft_consensus.cc:482] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fd90ec71d534cf096671d64b6337386)
I20260924 00:14:34.913061 8613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:14:34.913059 8614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:14:34.999781 8613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:34.999841 8614 raft_consensus.cc:504] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:35.000043 8614 leader_election.cc:288] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 2 election: Requested vote from peers 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919), 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189:45337)
I20260924 00:14:35.000041 8613 leader_election.cc:288] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [CANDIDATE]: Term 2 election: Requested vote from peers 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919), b002ba0cbd174921bc293680923e5a71 (127.1.44.188:45671)
I20260924 00:14:35.000416 7910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b002ba0cbd174921bc293680923e5a71" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
I20260924 00:14:35.000509 7910 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b002ba0cbd174921bc293680923e5a71 in current term 2: Already voted for candidate 6eaaaf9c3dbe440693c907ffec519dd3 in this term.
I20260924 00:14:35.000679 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.000218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45702)
0924 00:14:35.000613 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.000649 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374}
I20260924 00:14:35.000759 8104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b002ba0cbd174921bc293680923e5a71"
I20260924 00:14:35.000882 8104 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6eaaaf9c3dbe440693c907ffec519dd3 in current term 2: Already voted for candidate b002ba0cbd174921bc293680923e5a71 in this term.
I20260924 00:14:35.000950 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.000218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45704)
0924 00:14:35.000721 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.000753 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486}
I20260924 00:14:35.001327 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.001237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45722)
0924 00:14:35.001275 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.001306 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 00:14:35.001318 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.001200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:45710)
0924 00:14:35.001259 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.001289 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 00:14:35.001389 8081 leader_election.cc:334] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:35.001421 8081 leader_election.cc:302] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b002ba0cbd174921bc293680923e5a71; no voters: 1fd90ec71d534cf096671d64b6337386, 6eaaaf9c3dbe440693c907ffec519dd3
I20260924 00:14:35.001523 8614 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260924 00:14:35.001536 7887 leader_election.cc:334] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:35.001575 7887 leader_election.cc:302] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eaaaf9c3dbe440693c907ffec519dd3; no voters: 1fd90ec71d534cf096671d64b6337386, b002ba0cbd174921bc293680923e5a71
I20260924 00:14:35.001653 8613 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 00:14:35.406271 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.406016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:60659)
0924 00:14:35.406171 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.406211 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260924 00:14:35.408758 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.408589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:43221)
0924 00:14:35.408692 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.408724 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":67,"threads_started":1}
I20260924 00:14:35.409667 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:35.409486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:47745)
0924 00:14:35.409598 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:35.409634 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260924 00:14:36.407009 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:36.406790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:33965)
0924 00:14:36.406901 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:36.406939 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":61,"threads_started":1}
I20260924 00:14:36.409368 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:36.409192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:55257)
0924 00:14:36.409300 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:36.409335 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
I20260924 00:14:36.410214 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:36.410039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:51959)
0924 00:14:36.410151 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:36.410182 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260924 00:14:37.031491 8629 raft_consensus.cc:482] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:37.031584 8629 raft_consensus.cc:504] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:37.031759 8629 leader_election.cc:288] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919), 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189:45337)
I20260924 00:14:37.032001 7910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b002ba0cbd174921bc293680923e5a71" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" is_pre_election: true
I20260924 00:14:37.032095 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.031908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:46010)
0924 00:14:37.032008 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.032053 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":50,"threads_started":1}
I20260924 00:14:37.032117 7910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b002ba0cbd174921bc293680923e5a71 in term 2.
I20260924 00:14:37.032280 8078 leader_election.cc:302] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eaaaf9c3dbe440693c907ffec519dd3, b002ba0cbd174921bc293680923e5a71; no voters:
I20260924 00:14:37.032407 8629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 00:14:37.032461 8629 raft_consensus.cc:482] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:37.032488 8629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:14:37.032523 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.032415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:46022)
0924 00:14:37.032455 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.032499 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 00:14:37.032588 8081 leader_election.cc:334] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:37.033217 8629 raft_consensus.cc:504] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:37.033365 8629 leader_election.cc:288] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 3 election: Requested vote from peers 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919), 6eaaaf9c3dbe440693c907ffec519dd3 (127.1.44.189:45337)
I20260924 00:14:37.033510 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.033430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:46028)
0924 00:14:37.033462 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.033495 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 00:14:37.033504 7910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b002ba0cbd174921bc293680923e5a71" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6eaaaf9c3dbe440693c907ffec519dd3"
I20260924 00:14:37.033591 7910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:14:37.033798 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.033728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:46042)
0924 00:14:37.033754 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.033777 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260924 00:14:37.033905 8081 leader_election.cc:334] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
I20260924 00:14:37.034334 7910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b002ba0cbd174921bc293680923e5a71 in term 3.
I20260924 00:14:37.034551 8078 leader_election.cc:302] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6eaaaf9c3dbe440693c907ffec519dd3, b002ba0cbd174921bc293680923e5a71; no voters: 1fd90ec71d534cf096671d64b6337386
I20260924 00:14:37.034646 8629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 00:14:37.034765 8629 raft_consensus.cc:686] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [term 3 LEADER]: Becoming Leader. State: Replica: b002ba0cbd174921bc293680923e5a71, State: Running, Role: LEADER
I20260924 00:14:37.034884 8629 consensus_queue.cc:237] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } }
I20260924 00:14:37.035271 8632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b002ba0cbd174921bc293680923e5a71. Latest consensus state: current_term: 3 leader_uuid: "b002ba0cbd174921bc293680923e5a71" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:37.035374 8632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:37.035586 8634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:37.036154 8634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:37.036257 8634 catalog_manager.cc:1259] Loaded cluster ID: 7ff6785193474eb4b423f2110201643c
I20260924 00:14:37.036294 8634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:37.036473 8634 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:37.036536 8634 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Loaded TSK: 0
I20260924 00:14:37.036625 8634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:37.130146 8094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:37.131834 8094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:37.132845 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.132738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:46076)
0924 00:14:37.132773 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.132808 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260924 00:14:37.132876 7910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [term 3 FOLLOWER]: Refusing update from remote peer b002ba0cbd174921bc293680923e5a71: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260924 00:14:37.133158 8629 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 00:14:37.133237 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.133101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.0.0.1:46092)
0924 00:14:37.133190 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.133218 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260924 00:14:37.133351 8081 consensus_peers.cc:487] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 -> Peer 1fd90ec71d534cf096671d64b6337386 (127.1.44.190:37919): Couldn't send request to peer 1fd90ec71d534cf096671d64b6337386. Status: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 00:14:37.133790 8636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b002ba0cbd174921bc293680923e5a71. Latest consensus state: current_term: 3 leader_uuid: "b002ba0cbd174921bc293680923e5a71" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:37.133901 8636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:14:37.134469 8636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b002ba0cbd174921bc293680923e5a71" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:37.134796 8636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eaaaf9c3dbe440693c907ffec519dd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:14:37.134699 8637 mvcc.cc:205] Tried to move back new op lower bound from 7332695560734281728 to 7332695560335659008. Current Snapshot: MvccSnapshot[applied={T|T < 7332695560734281728}]
I20260924 00:14:37.134908 8632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b002ba0cbd174921bc293680923e5a71" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "1fd90ec71d534cf096671d64b6337386" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37919 } } peers { permanent_uuid: "6eaaaf9c3dbe440693c907ffec519dd3" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45337 } attrs { promote: false } } peers { permanent_uuid: "b002ba0cbd174921bc293680923e5a71" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 45671 } attrs { promote: false } } }
I20260924 00:14:37.134982 8632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:37.140270 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.136692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:44703 (local address 127.0.0.1:49012)
0924 00:14:37.136845 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.136855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:37.136907 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:37.137139 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:37.137139 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:37.137142 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:37.137188 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.137190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.138106 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.138107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.139436 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.139439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.140039 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.140040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.140097 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.140101 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:37.140154 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:14:37.140202 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":9,"thread_start_us":88,"threads_started":1}
I20260924 00:14:37.140270 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.136754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.1.44.129:44703)
0924 00:14:37.136995 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:37.136996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:37.137004 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:37.137031 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:37.137032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:37.137043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:37.137133 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:37.137365 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.138018 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.139557 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.139946 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.140037 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.140050 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:37.140164 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 00:14:37.140201 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":41,"threads_started":1}
I20260924 00:14:37.140635 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.136582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44951 (local address 127.0.0.1:34558)
0924 00:14:37.136819 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.136832 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:37.136882 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:37.137412 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:37.137413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:37.137414 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:37.137443 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.137444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.138291 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.138292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.139680 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.139682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.140442 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.140444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.140502 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.140506 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:37.140533 (+ 27us) client_negotiation.cc:240] Negotiation successful
0924 00:14:37.140570 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":336}
I20260924 00:14:37.140645 8207 tablet_service.cc:1467] Processing CreateTablet for tablet f5b0c6c7bbc64b1889253f026d32292a (DEFAULT_TABLE table=test-table [id=6c45f5ca04c04ccaa9bb2730f1bc7ad0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:37.140799 8207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b0c6c7bbc64b1889253f026d32292a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:37.140892 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.136893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34558 (local address 127.1.44.130:44951)
0924 00:14:37.137277 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:37.137279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:37.137286 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:37.137324 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:37.137325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:37.137335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:37.137398 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:37.137496 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.138212 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.139778 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.140345 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.140743 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.140758 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:37.140776 (+ 18us) server_negotiation.cc:299] Negotiation successful
0924 00:14:37.140825 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":54,"threads_started":1}
I20260924 00:14:37.140911 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.136710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:44661 (local address 127.0.0.1:59380)
0924 00:14:37.136879 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.136888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:37.136926 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:37.137251 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:37.137252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:37.137253 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:37.137285 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.137286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.138205 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.138206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.139387 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.139390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.140719 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.140720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.140773 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.140776 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:37.140820 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 00:14:37.140854 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260924 00:14:37.141176 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.137033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59380 (local address 127.1.44.131:44661)
0924 00:14:37.137133 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:37.137135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:37.137143 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:37.137167 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:37.137168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:37.137179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:37.137240 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:37.137352 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.138112 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.139828 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.140632 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.141055 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.141070 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:37.141095 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 00:14:37.141127 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":49,"threads_started":1}
I20260924 00:14:37.141395 8480 tablet_service.cc:1467] Processing CreateTablet for tablet f5b0c6c7bbc64b1889253f026d32292a (DEFAULT_TABLE table=test-table [id=6c45f5ca04c04ccaa9bb2730f1bc7ad0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:37.141511 8480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b0c6c7bbc64b1889253f026d32292a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:37.142246 8647 tablet_bootstrap.cc:492] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Bootstrap starting.
I20260924 00:14:37.142499 8344 tablet_service.cc:1467] Processing CreateTablet for tablet f5b0c6c7bbc64b1889253f026d32292a (DEFAULT_TABLE table=test-table [id=6c45f5ca04c04ccaa9bb2730f1bc7ad0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:37.142602 8344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b0c6c7bbc64b1889253f026d32292a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:37.142741 8647 tablet_bootstrap.cc:654] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:37.142974 8648 tablet_bootstrap.cc:492] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Bootstrap starting.
I20260924 00:14:37.143362 8648 tablet_bootstrap.cc:654] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:37.143481 8647 tablet_bootstrap.cc:492] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc: No bootstrap required, opened a new log
I20260924 00:14:37.143532 8647 ts_tablet_manager.cc:1397] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:37.143688 8647 raft_consensus.cc:348] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.143779 8647 raft_consensus.cc:374] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:37.143802 8647 raft_consensus.cc:729] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3d5ce1fbd4439db16d7c2123d1ebbc, State: Initialized, Role: FOLLOWER
I20260924 00:14:37.143853 8647 consensus_queue.cc:260] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.144023 8652 tablet_bootstrap.cc:492] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Bootstrap starting.
I20260924 00:14:37.144097 8647 ts_tablet_manager.cc:1428] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:37.144505 8652 tablet_bootstrap.cc:654] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:37.144575 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.144142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:49719)
0924 00:14:37.144487 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.144523 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":76,"threads_started":1}
I20260924 00:14:37.144887 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.144772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:39037)
0924 00:14:37.144829 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.144857 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 00:14:37.145030 8651 raft_consensus.cc:482] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:37.145104 8273 heartbeater.cc:503] Master 127.1.44.188:45671 was elected leader, sending a full tablet report...
I20260924 00:14:37.145080 8651 raft_consensus.cc:504] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.145725 8648 tablet_bootstrap.cc:492] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: No bootstrap required, opened a new log
I20260924 00:14:37.145785 8648 ts_tablet_manager.cc:1397] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 00:14:37.145810 8652 tablet_bootstrap.cc:492] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: No bootstrap required, opened a new log
I20260924 00:14:37.145855 8652 ts_tablet_manager.cc:1397] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:37.145915 8648 raft_consensus.cc:348] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.145987 8648 raft_consensus.cc:374] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:37.146013 8648 raft_consensus.cc:729] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161e7e7cf01244ea85e124b73c88d69c, State: Initialized, Role: FOLLOWER
I20260924 00:14:37.145994 8652 raft_consensus.cc:348] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.146056 8652 raft_consensus.cc:374] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:37.146083 8652 raft_consensus.cc:729] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1138b3d8b4bf4acba32cc3fb98079fb4, State: Initialized, Role: FOLLOWER
I20260924 00:14:37.146056 8648 consensus_queue.cc:260] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.146152 8652 consensus_queue.cc:260] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.146193 8648 ts_tablet_manager.cc:1428] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:37.146474 8652 ts_tablet_manager.cc:1428] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:37.146792 8651 leader_election.cc:288] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951), 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661)
I20260924 00:14:37.147001 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.146692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:50433)
0924 00:14:37.146936 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.146970 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":92,"threads_started":1}
I20260924 00:14:37.147277 8546 heartbeater.cc:503] Master 127.1.44.188:45671 was elected leader, sending a full tablet report...
I20260924 00:14:37.147343 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.147170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:52059)
0924 00:14:37.147297 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.147319 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260924 00:14:37.148197 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.148006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:55913)
0924 00:14:37.148116 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.148149 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":70,"threads_started":1}
I20260924 00:14:37.149212 8410 heartbeater.cc:503] Master 127.1.44.188:45671 was elected leader, sending a full tablet report...
I20260924 00:14:37.149456 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.148973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:59255)
0924 00:14:37.149404 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.149431 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417}
I20260924 00:14:37.149552 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.145436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44951 (local address 127.1.44.129:47591)
0924 00:14:37.145474 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.145484 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:37.145532 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:37.145656 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:37.145657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:37.145658 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:37.145711 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.145713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.146562 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.146563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.147785 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.147788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.149324 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.149325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.149396 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.149399 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:37.149444 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 00:14:37.149482 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 00:14:37.149631 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.145475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:47591 (local address 127.1.44.130:44951)
0924 00:14:37.145510 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:37.145512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:37.145519 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:37.145586 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:37.145588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:37.145597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:37.145653 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:37.145808 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.146497 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.148714 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.149211 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.149311 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.149320 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:37.149536 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 00:14:37.149568 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:37.149698 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.146266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:60075 (local address 127.1.44.131:44661)
0924 00:14:37.146310 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:37.146312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:37.146319 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:37.146338 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:37.146339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:37.146349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:37.146405 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:37.146791 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.147473 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.148699 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.149254 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.149330 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.149340 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:37.149594 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 00:14:37.149635 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 00:14:37.149780 8364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" is_pre_election: true
I20260924 00:14:37.149847 8364 raft_consensus.cc:2454] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a3d5ce1fbd4439db16d7c2123d1ebbc in term 0.
I20260924 00:14:37.149873 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.145460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:44661 (local address 127.1.44.129:60075)
0924 00:14:37.145563 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.145573 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:37.145618 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:37.146666 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:37.146667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:37.146669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:37.146712 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.146713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.147557 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.147558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.148579 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:37.148582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.149394 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.149395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:37.149464 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.149469 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:14:37.149521 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 00:14:37.149561 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":65,"threads_started":1}
I20260924 00:14:37.149999 8163 leader_election.cc:302] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1138b3d8b4bf4acba32cc3fb98079fb4, 2a3d5ce1fbd4439db16d7c2123d1ebbc; no voters:
I20260924 00:14:37.150099 8656 raft_consensus.cc:482] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:37.150101 8651 raft_consensus.cc:2781] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:37.150136 8651 raft_consensus.cc:482] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:37.150072 8500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "161e7e7cf01244ea85e124b73c88d69c" is_pre_election: true
I20260924 00:14:37.150168 8651 raft_consensus.cc:3037] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:37.150178 8656 raft_consensus.cc:504] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.150231 8500 raft_consensus.cc:2454] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a3d5ce1fbd4439db16d7c2123d1ebbc in term 0.
I20260924 00:14:37.150373 8364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" is_pre_election: true
I20260924 00:14:37.150439 8364 raft_consensus.cc:2454] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 161e7e7cf01244ea85e124b73c88d69c in term 0.
I20260924 00:14:37.150527 8656 leader_election.cc:288] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951), 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703)
I20260924 00:14:37.150602 8436 leader_election.cc:302] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1138b3d8b4bf4acba32cc3fb98079fb4, 161e7e7cf01244ea85e124b73c88d69c; no voters:
I20260924 00:14:37.150727 8227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" is_pre_election: true
I20260924 00:14:37.150873 8656 raft_consensus.cc:2781] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:37.150914 8656 raft_consensus.cc:482] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:37.150929 8656 raft_consensus.cc:3037] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:37.150977 8651 raft_consensus.cc:504] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.151088 8227 raft_consensus.cc:2379] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 161e7e7cf01244ea85e124b73c88d69c in current term 1: Already voted for candidate 2a3d5ce1fbd4439db16d7c2123d1ebbc in this term.
I20260924 00:14:37.151108 8651 leader_election.cc:288] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [CANDIDATE]: Term 1 election: Requested vote from peers 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951), 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661)
I20260924 00:14:37.151238 8364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
I20260924 00:14:37.151302 8364 raft_consensus.cc:3037] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:37.151292 8500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "161e7e7cf01244ea85e124b73c88d69c"
I20260924 00:14:37.151801 8364 raft_consensus.cc:2454] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a3d5ce1fbd4439db16d7c2123d1ebbc in term 1.
I20260924 00:14:37.151955 8163 leader_election.cc:302] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1138b3d8b4bf4acba32cc3fb98079fb4, 2a3d5ce1fbd4439db16d7c2123d1ebbc; no voters:
I20260924 00:14:37.152100 8651 raft_consensus.cc:2781] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:37.152238 8651 raft_consensus.cc:686] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [term 1 LEADER]: Becoming Leader. State: Replica: 2a3d5ce1fbd4439db16d7c2123d1ebbc, State: Running, Role: LEADER
I20260924 00:14:37.152331 8651 consensus_queue.cc:237] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.152477 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.152394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:47765)
0924 00:14:37.152429 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.152451 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 00:14:37.152689 8656 raft_consensus.cc:504] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } }
I20260924 00:14:37.152827 8500 raft_consensus.cc:2379] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2a3d5ce1fbd4439db16d7c2123d1ebbc in current term 1: Already voted for candidate 161e7e7cf01244ea85e124b73c88d69c in this term.
I20260924 00:14:37.153349 8094 catalog_manager.cc:5495] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129). New cstate: current_term: 1 leader_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 44703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:37.153874 8364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4"
I20260924 00:14:37.153957 8364 raft_consensus.cc:2379] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 161e7e7cf01244ea85e124b73c88d69c in current term 1: Already voted for candidate 2a3d5ce1fbd4439db16d7c2123d1ebbc in this term.
I20260924 00:14:37.154114 8656 leader_election.cc:288] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 election: Requested vote from peers 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951), 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703)
I20260924 00:14:37.154295 8227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b0c6c7bbc64b1889253f026d32292a" candidate_uuid: "161e7e7cf01244ea85e124b73c88d69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a3d5ce1fbd4439db16d7c2123d1ebbc"
I20260924 00:14:37.154577 8436 leader_election.cc:302] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 161e7e7cf01244ea85e124b73c88d69c; no voters: 1138b3d8b4bf4acba32cc3fb98079fb4, 2a3d5ce1fbd4439db16d7c2123d1ebbc
I20260924 00:14:37.154667 8656 raft_consensus.cc:2726] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260924 00:14:37.156595 8549 tablet.cc:2369] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:37.160761 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.158730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.1.44.129:44703)
0924 00:14:37.158780 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:37.158782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:37.158790 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:37.158821 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:37.158822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:37.158836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:37.158890 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:37.158986 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.159454 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.160124 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:37.160331 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:37.160380 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:37.160536 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:37.160619 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:37.160669 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:37.160709 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 00:14:37.160722 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:14:37.161581 8364 raft_consensus.cc:1261] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Refusing update from remote peer 2a3d5ce1fbd4439db16d7c2123d1ebbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:37.161628 8500 raft_consensus.cc:1261] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Refusing update from remote peer 2a3d5ce1fbd4439db16d7c2123d1ebbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:37.161789 8661 consensus_queue.cc:1035] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1138b3d8b4bf4acba32cc3fb98079fb4" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:37.161897 8651 consensus_queue.cc:1035] T f5b0c6c7bbc64b1889253f026d32292a P 2a3d5ce1fbd4439db16d7c2123d1ebbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "161e7e7cf01244ea85e124b73c88d69c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 44661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:37.162552 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.162456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:59799)
0924 00:14:37.162491 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.162523 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 00:14:37.162811 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.162703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:44193)
0924 00:14:37.162744 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.162780 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260924 00:14:37.163733 8664 mvcc.cc:205] Tried to move back new op lower bound from 7332695560852357120 to 7332695560816484352. Current Snapshot: MvccSnapshot[applied={T|T < 7332695560852357120}]
I20260924 00:14:37.164409 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.164220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:48827)
0924 00:14:37.164353 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.164383 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260924 00:14:37.164443 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.164341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.131:38383)
0924 00:14:37.164397 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.164419 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 00:14:37.164775 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.164694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.130:54701)
0924 00:14:37.164724 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.164749 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 00:14:37.165268 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.165193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37919 (local address 127.1.44.129:38965)
0924 00:14:37.165222 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.165247 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260924 00:14:37.166581 8094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:37.166671 8094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:37.167698 8094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b002ba0cbd174921bc293680923e5a71: Sending DeleteTablet for 3 replicas of tablet f5b0c6c7bbc64b1889253f026d32292a
I20260924 00:14:37.167981 8344 tablet_service.cc:1514] Processing DeleteTablet for tablet f5b0c6c7bbc64b1889253f026d32292a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:37 UTC) from {username='oryx', principal='oryx/127.1.44.188@KRBTEST.COM'} at 127.0.0.1:34558
I20260924 00:14:37.167981 8207 tablet_service.cc:1514] Processing DeleteTablet for tablet f5b0c6c7bbc64b1889253f026d32292a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:37 UTC) from {username='oryx', principal='oryx/127.1.44.188@KRBTEST.COM'} at 127.0.0.1:49012
I20260924 00:14:37.168138 8480 tablet_service.cc:1514] Processing DeleteTablet for tablet f5b0c6c7bbc64b1889253f026d32292a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:37 UTC) from {username='oryx', principal='oryx/127.1.44.188@KRBTEST.COM'} at 127.0.0.1:59380
I20260924 00:14:37.168154 8675 tablet_replica.cc:330] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: stopping tablet replica
I20260924 00:14:37.168205 8675 raft_consensus.cc:2229] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:37.168252 8675 raft_consensus.cc:2258] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:37.168536 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 8051
I20260924 00:14:37.169912 8675 ts_tablet_manager.cc:1905] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:37.171329 8675 ts_tablet_manager.cc:1918] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:37.171382 8675 log.cc:1196] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/f5b0c6c7bbc64b1889253f026d32292a
I20260924 00:14:37.171605 8675 ts_tablet_manager.cc:1939] T f5b0c6c7bbc64b1889253f026d32292a P 1138b3d8b4bf4acba32cc3fb98079fb4: Deleting consensus metadata
I20260924 00:14:37.171891 8081 catalog_manager.cc:4841] TS 1138b3d8b4bf4acba32cc3fb98079fb4 (127.1.44.130:44951): tablet f5b0c6c7bbc64b1889253f026d32292a (table test-table [id=6c45f5ca04c04ccaa9bb2730f1bc7ad0]) successfully deleted
I20260924 00:14:37.174513 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:37.174417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:44703 (local address 127.0.0.1:49040)
0924 00:14:37.174465 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:37.174495 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 00:14:37.174642 8676 tablet_replica.cc:330] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: stopping tablet replica
W20260924 00:14:37.174671 8081 catalog_manager.cc:4584] TS 2a3d5ce1fbd4439db16d7c2123d1ebbc (127.1.44.129:44703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5b0c6c7bbc64b1889253f026d32292a: Network error: Client connection negotiation failed: client connection to 127.1.44.129:44703: connect: Connection refused (error 111)
I20260924 00:14:37.174700 8676 raft_consensus.cc:2229] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:37.174739 8676 raft_consensus.cc:2258] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:37.174809 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 8281
I20260924 00:14:37.175524 8676 ts_tablet_manager.cc:1905] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:37.177286 8676 ts_tablet_manager.cc:1918] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:37.177346 8676 log.cc:1196] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/f5b0c6c7bbc64b1889253f026d32292a
I20260924 00:14:37.177578 8676 ts_tablet_manager.cc:1939] T f5b0c6c7bbc64b1889253f026d32292a P 161e7e7cf01244ea85e124b73c88d69c: Deleting consensus metadata
I20260924 00:14:37.178049 8080 catalog_manager.cc:4841] TS 161e7e7cf01244ea85e124b73c88d69c (127.1.44.131:44661): tablet f5b0c6c7bbc64b1889253f026d32292a (table test-table [id=6c45f5ca04c04ccaa9bb2730f1bc7ad0]) successfully deleted
I20260924 00:14:37.181046 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 8417
I20260924 00:14:37.186597 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7767
I20260924 00:14:37.191089 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 7954
2026-09-24T00:14:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6774 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:37 dist-test-slave-cm7b krb5kdc[8679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:37 dist-test-slave-cm7b krb5kdc[8679](info): set up 2 sockets
Sep 24 00:14:37 dist-test-slave-cm7b krb5kdc[8679](info): commencing operation
krb5kdc: starting...
W20260924 00:14:38.227598 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab.
Entry for principal oryx/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab.
I20260924 00:14:38.338362 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44167
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab
--principal=oryx/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:38.414407 8692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:38.414623 8692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:38.414659 8692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:38.415784 8692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:38.415844 8692 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 00:14:38.415867 8692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:38.415889 8692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:38.415917 8692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:38.415941 8692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:38.417371 8692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.190.keytab
--principal=oryx/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:38.417623 8692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:38.417827 8692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:38.422573 8698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:38.422590 8700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:38.422713 8692 server_base.cc:1034] running on GCE node
W20260924 00:14:38.422590 8697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:38.423009 8692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:38.423331 8692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:38.424495 8692 hybrid_clock.cc:648] HybridClock initialized: now 1790208878424481 us; error 29 us; skew 500 ppm
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:38.426872 8692 init.cc:377] Logged in from keytab as oryx/127.1.44.190@KRBTEST.COM (short username oryx)
I20260924 00:14:38.427346 8692 webserver.cc:460] Webserver started at http://127.1.44.190:36015/ using document root <none> and password file <none>
I20260924 00:14:38.427582 8692 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:38.427626 8692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:38.427768 8692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:38.428788 8692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "753e6d1a74db4d0bb52d0fe267d66f30"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "8b0ba6382c1740b39052d09172ebb735"
server_key_iv: "768a1558223208b18130ca45ea5bbcef"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.429144 8692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "753e6d1a74db4d0bb52d0fe267d66f30"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "8b0ba6382c1740b39052d09172ebb735"
server_key_iv: "768a1558223208b18130ca45ea5bbcef"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.430380 8692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:38.431099 8707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.431309 8692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:38.431401 8692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "753e6d1a74db4d0bb52d0fe267d66f30"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "8b0ba6382c1740b39052d09172ebb735"
server_key_iv: "768a1558223208b18130ca45ea5bbcef"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.431489 8692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:38.444380 8692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:38.446040 8692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:38.446141 8692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:38.450668 8692 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44167
I20260924 00:14:38.450716 8769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44167 every 8 connection(s)
I20260924 00:14:38.451000 8692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:38.451582 8770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:38.451906 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 8692
I20260924 00:14:38.452030 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:38.452231 1202 external_mini_cluster.cc:1428] Setting key a1218c12063d6a99ba78fabb58c19d1f
I20260924 00:14:38.454476 8770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Bootstrap starting.
I20260924 00:14:38.455152 8770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:38.455459 8770 log.cc:824] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:38.456111 8770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: No bootstrap required, opened a new log
I20260924 00:14:38.458103 8770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44167 } }
I20260924 00:14:38.458225 8770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:38.458247 8770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753e6d1a74db4d0bb52d0fe267d66f30, State: Initialized, Role: FOLLOWER
I20260924 00:14:38.458381 8770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44167 } }
I20260924 00:14:38.458457 8770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:38.458479 8770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:38.458544 8770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 0 FOLLOWER]: Advancing to term 1
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:38.459093 8770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44167 } }
I20260924 00:14:38.459230 8770 leader_election.cc:302] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 753e6d1a74db4d0bb52d0fe267d66f30; no voters:
I20260924 00:14:38.459389 8770 leader_election.cc:288] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:38.459425 8775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:38.459533 8775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [term 1 LEADER]: Becoming Leader. State: Replica: 753e6d1a74db4d0bb52d0fe267d66f30, State: Running, Role: LEADER
I20260924 00:14:38.459631 8770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:38.459605 8775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44167 } }
I20260924 00:14:38.459950 8777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 753e6d1a74db4d0bb52d0fe267d66f30. Latest consensus state: current_term: 1 leader_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44167 } } }
I20260924 00:14:38.459954 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753e6d1a74db4d0bb52d0fe267d66f30" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44167 } } }
I20260924 00:14:38.460048 8777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:38.460063 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:38.460649 8781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:38.461163 8781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:38.462710 8781 catalog_manager.cc:1347] Generated new cluster ID: 2d329f18298644dfaf324a336e64da55
I20260924 00:14:38.463485 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.452724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.1.44.190:44167)
0924 00:14:38.452917 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.452926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.452960 (+ 34us) server_negotiation.cc:407] Connection header received
0924 00:14:38.453480 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.453487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.453543 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.453691 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.454248 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.455579 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.456431 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.457081 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.457214 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.459571 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.459576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.459576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.459589 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.461918 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.462308 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.462311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.462312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.462406 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.462609 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.462609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.462610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.462873 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.463030 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.463145 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.463301 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
I20260924 00:14:38.463614 8781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:38.473667 8781 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:38.474128 8781 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:38.480217 8781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Generated new TSK 0
I20260924 00:14:38.480378 8781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab.
Entry for principal oryx/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab.
I20260924 00:14:38.506829 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44167
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:38.584568 8796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:38.584753 8796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:38.584777 8796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:38.585834 8796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:38.585865 8796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:38.585886 8796 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:38.585902 8796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:38.587288 8796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:38.587459 8796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:38.587613 8796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:38.592161 8804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:38.592236 8796 server_base.cc:1034] running on GCE node
W20260924 00:14:38.592200 8801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:38.592203 8802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:38.592535 8796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:38.592722 8796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:38.593856 8796 hybrid_clock.cc:648] HybridClock initialized: now 1790208878593837 us; error 36 us; skew 500 ppm
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:38.596288 8796 init.cc:377] Logged in from keytab as oryx/127.1.44.129@KRBTEST.COM (short username oryx)
I20260924 00:14:38.596632 8796 webserver.cc:460] Webserver started at http://127.1.44.129:32819/ using document root <none> and password file <none>
I20260924 00:14:38.596793 8796 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:38.596832 8796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:38.596915 8796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:38.597831 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "817f7e26786a4c1e9956f59df649bf46"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "32fdcef1038fd5675b95323e2d790fe8"
server_key_iv: "bd5023bca4a2df48743b9d23b6a64c3a"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.598110 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "817f7e26786a4c1e9956f59df649bf46"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "32fdcef1038fd5675b95323e2d790fe8"
server_key_iv: "bd5023bca4a2df48743b9d23b6a64c3a"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.599298 8796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:38.599951 8811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.600106 8796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:38.600201 8796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "817f7e26786a4c1e9956f59df649bf46"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "32fdcef1038fd5675b95323e2d790fe8"
server_key_iv: "bd5023bca4a2df48743b9d23b6a64c3a"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.600278 8796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:38.609624 8796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:38.611742 8796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:38.611891 8796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:38.612344 8796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:38.612401 8796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.612471 8796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:38.612504 8796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.618724 8796 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:36165
I20260924 00:14:38.618794 8926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:36165 every 8 connection(s)
I20260924 00:14:38.619088 8796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:38.620667 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 8796
I20260924 00:14:38.620898 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:38.621104 1202 external_mini_cluster.cc:1428] Setting key 18d7e4db29a5ff4d71bf1814075325c2
I20260924 00:14:38.622668 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.612822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:38417 (local address 127.1.44.190:44167)
0924 00:14:38.612894 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.612897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.613068 (+ 171us) server_negotiation.cc:407] Connection header received
0924 00:14:38.613526 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.613527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.613542 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.613625 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.614414 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.615274 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.616796 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.617138 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.617225 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.619730 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.619731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.619731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.619738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.621305 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.621743 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.621744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.621745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.621807 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.622108 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.622108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.622108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.622203 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.622251 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.622548 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.622610 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 00:14:38.622985 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.612807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.1.44.129:38417)
0924 00:14:38.612948 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.612984 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.613438 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.613670 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.613673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.614029 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:38.614310 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.614316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.615373 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.615375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.616699 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.616702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.617250 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.617252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.617363 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.617896 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:38.617907 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:38.619646 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:38.621384 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.621391 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.621393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.621670 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.621903 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.621903 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.621904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.622044 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.622262 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:38.622267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:38.622423 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.622533 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.622696 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260924 00:14:38.623498 8927 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44167
I20260924 00:14:38.623580 8927 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:38.623747 8927 heartbeater.cc:511] Master 127.1.44.190:44167 requested a full tablet report, sending...
I20260924 00:14:38.624243 8723 ts_manager.cc:194] Registered new tserver with Master: 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165)
I20260924 00:14:38.624940 8723 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:38417
WARNING: no policy specified for oryx/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.130@KRBTEST.COM" created.
I20260924 00:14:38.633354 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.624446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.1.44.129:46279)
0924 00:14:38.627291 (+ 2845us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.627302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.627369 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.627488 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.627489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.627670 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:38.627726 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.627727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.628350 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.628351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.628895 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.628897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.629232 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.629233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.629292 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.629726 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:38.629731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:38.630008 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:38.632361 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.632361 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.632362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.632522 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.632739 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.632739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.632740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.632800 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.633030 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:38.633031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:38.633074 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.633156 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.633250 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2819,"thread_start_us":77,"threads_started":1}
I20260924 00:14:38.633364 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.624494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:46279 (local address 127.1.44.190:44167)
0924 00:14:38.624526 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.624527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.627358 (+ 2831us) server_negotiation.cc:407] Connection header received
0924 00:14:38.627432 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.627433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.627442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.627486 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.627800 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.628255 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.628969 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.629160 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.629234 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.630099 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.630100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.630100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.630106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.631352 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.632619 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.632620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.632620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.632670 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.632864 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.632865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.632865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.632968 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.633037 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.633214 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.633281 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab.
Entry for principal oryx/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab.
I20260924 00:14:38.644345 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44167
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:38.720986 8936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:38.721194 8936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:38.721235 8936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:38.722359 8936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:38.722424 8936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:38.722458 8936 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:38.722515 8936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:38.724102 8936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:38.724347 8936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:38.724566 8936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:38.729321 8942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:38.729352 8941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:38.729321 8944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:38.729671 8936 server_base.cc:1034] running on GCE node
I20260924 00:14:38.729926 8936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:38.730171 8936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:38.731309 8936 hybrid_clock.cc:648] HybridClock initialized: now 1790208878731303 us; error 38 us; skew 500 ppm
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:38.733742 8936 init.cc:377] Logged in from keytab as oryx/127.1.44.130@KRBTEST.COM (short username oryx)
I20260924 00:14:38.734220 8936 webserver.cc:460] Webserver started at http://127.1.44.130:44001/ using document root <none> and password file <none>
I20260924 00:14:38.734503 8936 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:38.734570 8936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:38.734699 8936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:38.735780 8936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "f4d4a51dd07544089996f623e2c9ebf3"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "48a8ffe8e5284b102c5ad34f7e1ab5a1"
server_key_iv: "e01e7a29490ecc2017c56f7bb42100b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.736135 8936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "f4d4a51dd07544089996f623e2c9ebf3"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "48a8ffe8e5284b102c5ad34f7e1ab5a1"
server_key_iv: "e01e7a29490ecc2017c56f7bb42100b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.737388 8936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:38.738128 8951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.738299 8936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:38.738395 8936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "f4d4a51dd07544089996f623e2c9ebf3"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "48a8ffe8e5284b102c5ad34f7e1ab5a1"
server_key_iv: "e01e7a29490ecc2017c56f7bb42100b5"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.738477 8936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:38.756510 8936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:38.758641 8936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:38.758793 8936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:38.759347 8936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:38.759397 8936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.759424 8936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:38.759447 8936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:38.766601 8936 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:36657
I20260924 00:14:38.766920 8936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:38.766978 9066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:36657 every 8 connection(s)
I20260924 00:14:38.768369 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 8936
I20260924 00:14:38.768491 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:38.768709 1202 external_mini_cluster.cc:1428] Setting key 6282d5c2cf02613a0670f96554309f8b
I20260924 00:14:38.769063 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.759701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:50287 (local address 127.1.44.190:44167)
0924 00:14:38.759742 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.759744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.760033 (+ 289us) server_negotiation.cc:407] Connection header received
0924 00:14:38.760586 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.760587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.760599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.760657 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.761318 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.761824 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.762793 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.763018 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.763096 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.766166 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.766168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.766168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.766176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.767621 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.768071 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.768072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.768073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.768133 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.768401 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.768402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.768402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.768559 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.768621 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.768900 (+ 279us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.768974 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:38.769253 8960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.759717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.1.44.130:50287)
0924 00:14:38.759916 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.759955 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.760487 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.760764 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.760768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.761048 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:38.761211 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.761215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.761916 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.761917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.762695 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.762697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.763100 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.763101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.763285 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.763810 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:38.763820 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:38.766072 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:38.767706 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.767712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.767714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.767978 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.768198 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.768199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.768199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.768322 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.768612 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:38.768617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:38.768750 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.768879 (+ 129us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.769026 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
I20260924 00:14:38.769699 9067 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44167
I20260924 00:14:38.769778 9067 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:38.769961 9067 heartbeater.cc:511] Master 127.1.44.190:44167 requested a full tablet report, sending...
I20260924 00:14:38.770305 8724 ts_manager.cc:194] Registered new tserver with Master: f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657)
I20260924 00:14:38.770727 8724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:50287
I20260924 00:14:38.776784 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.770835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:57433 (local address 127.1.44.190:44167)
0924 00:14:38.770922 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.770924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.771058 (+ 134us) server_negotiation.cc:407] Connection header received
0924 00:14:38.771130 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.771131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.771140 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.771188 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.771525 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.771975 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.773066 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.773259 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.773338 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.774304 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.774305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.774305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.774311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.775655 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.776059 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.776060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.776060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.776113 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.776322 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.776323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.776323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.776445 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.776496 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.776673 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.776731 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 00:14:38.776877 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.770667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.1.44.130:57433)
0924 00:14:38.770977 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.770987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.771073 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.771208 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.771208 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.771385 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:38.771436 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.771438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.772066 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.772067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.772977 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.772979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.773349 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.773350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.773436 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.773789 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:38.773794 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:38.774213 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:38.775731 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.775732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.775733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.775968 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.776187 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.776188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.776188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.776256 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.776511 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:38.776512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:38.776558 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.776649 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.776748 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab.
Entry for principal oryx/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab.
I20260924 00:14:38.794242 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44167
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:38.872584 9076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:38.872790 9076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:38.872828 9076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:38.873931 9076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:38.873993 9076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:38.874027 9076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 00:14:38.874068 9076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:38.875502 9076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.44.190:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:38.875741 9076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:38.875968 9076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:14:38.880594 9076 server_base.cc:1034] running on GCE node
W20260924 00:14:38.880617 9081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:38.880724 9082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:38.880628 9084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:38.881024 9076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:38.881244 9076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:38.882402 9076 hybrid_clock.cc:648] HybridClock initialized: now 1790208878882392 us; error 41 us; skew 500 ppm
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:38.884801 9076 init.cc:377] Logged in from keytab as oryx/127.1.44.131@KRBTEST.COM (short username oryx)
I20260924 00:14:38.885243 9076 webserver.cc:460] Webserver started at http://127.1.44.131:35295/ using document root <none> and password file <none>
I20260924 00:14:38.885448 9076 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:38.885512 9076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:38.885654 9076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:38.886590 9076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "9155954f4ebb4ddc8e2f218d563616e6"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "532dd01301757fc3dd4b5c56d484e0b7"
server_key_iv: "a41745d6713e02dfa7686e3d65985622"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.886930 9076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "9155954f4ebb4ddc8e2f218d563616e6"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "532dd01301757fc3dd4b5c56d484e0b7"
server_key_iv: "a41745d6713e02dfa7686e3d65985622"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.888217 9076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:38.888873 9091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.889043 9076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:38.889133 9076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "9155954f4ebb4ddc8e2f218d563616e6"
format_stamp: "Formatted at 2026-09-24 00:14:38 on dist-test-slave-cm7b"
server_key: "532dd01301757fc3dd4b5c56d484e0b7"
server_key_iv: "a41745d6713e02dfa7686e3d65985622"
server_key_version: "encryptionkey@0"
I20260924 00:14:38.889214 9076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:38.927793 9076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:38.929633 9076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:38.929787 9076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:38.930246 9076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:38.930308 9076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:38.930359 9076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:38.930385 9076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:38.937469 9076 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:34931
I20260924 00:14:38.937548 9206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:34931 every 8 connection(s)
I20260924 00:14:38.937836 9076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:38.938455 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9076
I20260924 00:14:38.938591 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:38.938789 1202 external_mini_cluster.cc:1428] Setting key 7907fa392b5f55e9f761767cfeaeca9d
I20260924 00:14:38.940610 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.930761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:43533 (local address 127.1.44.190:44167)
0924 00:14:38.930838 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.930841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.931000 (+ 159us) server_negotiation.cc:407] Connection header received
0924 00:14:38.931579 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.931581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.931592 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.931647 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.932281 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.933112 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.934745 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.935063 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.935139 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.937619 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.937621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.937621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.937628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.939354 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.939852 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.939853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.939854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.939906 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.940091 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.940091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.940092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.940188 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.940232 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.940496 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.940553 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 00:14:38.940807 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.930689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.1.44.131:43533)
0924 00:14:38.930874 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.930911 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.931474 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.931713 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.931717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.932011 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:38.932179 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.932183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.933210 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.933211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.934644 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.934648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.935172 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.935174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.935364 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.935819 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:38.935831 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:38.937505 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:38.939439 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.939445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.939447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.939766 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.939949 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.939949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.939950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.940036 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.940230 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:38.940232 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:38.940376 (+ 144us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.940470 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.940580 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260924 00:14:38.941254 9207 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44167
I20260924 00:14:38.941349 9207 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:38.941519 9207 heartbeater.cc:511] Master 127.1.44.190:44167 requested a full tablet report, sending...
I20260924 00:14:38.941883 8723 ts_manager.cc:194] Registered new tserver with Master: 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931)
I20260924 00:14:38.942408 8723 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:43533
I20260924 00:14:38.947353 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.942265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.1.44.131:46463)
0924 00:14:38.942379 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.942389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.942446 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.942598 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.942598 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.942794 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:38.942903 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.942904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.943608 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.943609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.944112 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.944113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.944476 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.944477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.944539 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.944901 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:38.944905 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:38.945146 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:38.946325 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.946326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.946327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.946491 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.946694 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:38.946695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:38.946696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:38.946751 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:38.947014 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:38.947016 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:38.947057 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.947142 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.947252 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":68,"threads_started":1}
I20260924 00:14:38.947340 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.942324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:46463 (local address 127.1.44.190:44167)
0924 00:14:38.942360 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.942362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.942430 (+ 68us) server_negotiation.cc:407] Connection header received
0924 00:14:38.942508 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.942509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.942517 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.942585 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.942984 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.943520 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.944208 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.944398 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.944466 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.945231 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.945232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.945233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.945239 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.946247 (+ 1008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.946575 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.946576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.946577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.946631 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.946822 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.946823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.946823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.946955 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.947012 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.947166 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.947252 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:38.950331 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:14:38.970503 8773 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:38.965367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.1.44.190:44167)
0924 00:14:38.965429 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.965431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.965439 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:38.965458 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.965459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.965470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.965521 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.966057 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.966504 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.967408 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.967579 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.967632 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.970458 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37516: BlockingRecv error: recv got EOF from 127.0.0.1:37516 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 00:14:38 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:38.977586 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.971413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.1.44.190:44167)
0924 00:14:38.971451 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.971452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.971517 (+ 65us) server_negotiation.cc:407] Connection header received
0924 00:14:38.971558 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.971558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.971565 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.971610 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:38.972025 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.972444 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.973062 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.973225 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.973274 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.975151 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:38.975152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:38.975152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:38.975159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:38.976517 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.976870 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.976872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.976872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.976929 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:38.977124 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:38.977125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:38.977125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:38.977228 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:38.977327 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.977427 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.977494 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:38.979159 8723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:38.980882 8723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:38.986918 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.983181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36165 (local address 127.0.0.1:56238)
0924 00:14:38.983386 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.983396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.983447 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.983964 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.983965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.983968 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:38.984012 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.984014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.984859 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.984860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.985883 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.985885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.986719 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.986720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.986771 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.986782 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.986834 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.986869 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":56,"thread_start_us":79,"threads_started":1}
I20260924 00:14:38.986977 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.983368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.1.44.129:36165)
0924 00:14:38.983730 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.983735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.983753 (+ 18us) server_negotiation.cc:407] Connection header received
0924 00:14:38.983791 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.983793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.983808 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.983892 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:38.984106 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.984765 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.985973 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.986640 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.986714 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.986786 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.986847 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.986883 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":87,"threads_started":1}
I20260924 00:14:38.987670 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.982994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:36657 (local address 127.0.0.1:41280)
0924 00:14:38.983344 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.983354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.983414 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.983622 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.983623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.983624 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:38.983682 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.983684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.984934 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.984934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.986130 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.986132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.987423 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.987424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.987488 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.987492 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.987567 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.987604 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":4,"thread_start_us":73,"threads_started":1}
I20260924 00:14:38.987725 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.983157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.1.44.130:36657)
0924 00:14:38.983328 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.983346 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.983482 (+ 136us) server_negotiation.cc:407] Connection header received
0924 00:14:38.983522 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.983524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.983539 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.983612 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:38.983775 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.984823 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.986215 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.986965 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.987040 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.987113 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.987580 (+ 467us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.987625 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260924 00:14:38.987772 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 895dd3d3b65e41d99658137179e55523 (DEFAULT_TABLE table=test-table [id=41ea1980ace24522841621b37cbb49c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:38.988170 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895dd3d3b65e41d99658137179e55523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:38.988370 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.982962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34931 (local address 127.0.0.1:33758)
0924 00:14:38.983084 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:38.983123 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:38.983179 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:38.983569 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:38.983571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:38.983573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:38.983646 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.983648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.984931 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.984932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.986131 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:38.986132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.988145 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.988146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:38.988210 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.988213 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:38.988270 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:14:38.988307 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":55,"thread_start_us":70,"threads_started":1}
I20260924 00:14:38.988420 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:38.983118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33758 (local address 127.1.44.131:34931)
0924 00:14:38.983414 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:38.983418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:38.983428 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:38.983471 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:38.983474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:38.983489 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:38.983571 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:38.983736 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.984807 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.987151 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:38.987958 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:38.988028 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:38.988112 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:38.988286 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 00:14:38.988322 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":104,"threads_started":1}
I20260924 00:14:38.989012 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 895dd3d3b65e41d99658137179e55523 (DEFAULT_TABLE table=test-table [id=41ea1980ace24522841621b37cbb49c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:38.989324 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895dd3d3b65e41d99658137179e55523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:38.991509 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 895dd3d3b65e41d99658137179e55523 (DEFAULT_TABLE table=test-table [id=41ea1980ace24522841621b37cbb49c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:38.991858 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895dd3d3b65e41d99658137179e55523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:38.992077 9235 tablet_bootstrap.cc:492] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Bootstrap starting.
I20260924 00:14:38.992828 9235 tablet_bootstrap.cc:654] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:38.993213 9235 log.cc:824] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:38.993544 9236 tablet_bootstrap.cc:492] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Bootstrap starting.
I20260924 00:14:38.993929 9235 tablet_bootstrap.cc:492] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: No bootstrap required, opened a new log
I20260924 00:14:38.994012 9235 ts_tablet_manager.cc:1397] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:38.994161 9236 tablet_bootstrap.cc:654] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:38.994508 9236 log.cc:824] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:38.995303 9236 tablet_bootstrap.cc:492] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: No bootstrap required, opened a new log
I20260924 00:14:38.995368 9236 ts_tablet_manager.cc:1397] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:38.995599 9239 tablet_bootstrap.cc:492] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Bootstrap starting.
I20260924 00:14:38.996018 9235 raft_consensus.cc:348] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:38.996150 9239 tablet_bootstrap.cc:654] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:38.996168 9235 raft_consensus.cc:374] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:38.996203 9235 raft_consensus.cc:729] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Initialized, Role: FOLLOWER
I20260924 00:14:38.996320 9235 consensus_queue.cc:260] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:38.996413 9239 log.cc:824] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:38.996574 9235 ts_tablet_manager.cc:1428] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:14:38.996625 8927 heartbeater.cc:503] Master 127.1.44.190:44167 was elected leader, sending a full tablet report...
I20260924 00:14:38.997098 9239 tablet_bootstrap.cc:492] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: No bootstrap required, opened a new log
I20260924 00:14:38.997152 9239 ts_tablet_manager.cc:1397] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:14:38.997359 9236 raft_consensus.cc:348] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:38.997503 9236 raft_consensus.cc:374] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:38.997534 9236 raft_consensus.cc:729] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9155954f4ebb4ddc8e2f218d563616e6, State: Initialized, Role: FOLLOWER
I20260924 00:14:38.997649 9236 consensus_queue.cc:260] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:38.997874 9236 ts_tablet_manager.cc:1428] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:14:38.997916 9207 heartbeater.cc:503] Master 127.1.44.190:44167 was elected leader, sending a full tablet report...
I20260924 00:14:38.999027 9239 raft_consensus.cc:348] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:38.999166 9239 raft_consensus.cc:374] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:38.999202 9239 raft_consensus.cc:729] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Initialized, Role: FOLLOWER
I20260924 00:14:38.999332 9239 consensus_queue.cc:260] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:38.999575 9239 ts_tablet_manager.cc:1428] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 00:14:38.999635 9067 heartbeater.cc:503] Master 127.1.44.190:44167 was elected leader, sending a full tablet report...
I20260924 00:14:39.001614 9243 raft_consensus.cc:482] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:39.001677 9243 raft_consensus.cc:504] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.001965 9243 leader_election.cc:288] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931), 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165)
I20260924 00:14:39.004858 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.002257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:55653 (local address 127.1.44.131:34931)
0924 00:14:39.002299 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.002301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.002366 (+ 65us) server_negotiation.cc:407] Connection header received
0924 00:14:39.002440 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.002441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.002449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.002501 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:39.002648 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.003094 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.004162 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.004514 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.004605 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.004612 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.004766 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.004798 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 00:14:39.004902 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.002162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:57585 (local address 127.1.44.129:36165)
0924 00:14:39.002202 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.002204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.002212 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:39.002233 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.002234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.002244 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.002300 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:39.002470 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.002972 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.004175 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.004541 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.004632 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.004640 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.004802 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.004833 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:14:39.004927 8960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.002095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36165 (local address 127.1.44.130:57585)
0924 00:14:39.002140 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.002150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.002184 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.002339 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.002340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.002341 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:39.002385 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.002386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.003058 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.003058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.004077 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.004080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.004633 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.004633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.004710 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.004713 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.004794 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.004833 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 00:14:39.004956 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.002107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34931 (local address 127.1.44.130:55653)
0924 00:14:39.002318 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.002327 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.002374 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.002505 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.002506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.002507 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:39.002572 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.002574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.003182 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.003183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.004081 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.004084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.004607 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.004608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.004682 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.004685 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.004759 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.004795 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":82,"threads_started":1}
I20260924 00:14:39.005275 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895dd3d3b65e41d99658137179e55523" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6" is_pre_election: true
I20260924 00:14:39.005368 9161 raft_consensus.cc:2454] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 0.
I20260924 00:14:39.005406 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895dd3d3b65e41d99658137179e55523" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817f7e26786a4c1e9956f59df649bf46" is_pre_election: true
I20260924 00:14:39.005520 8881 raft_consensus.cc:2454] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 0.
I20260924 00:14:39.005574 8952 leader_election.cc:302] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9155954f4ebb4ddc8e2f218d563616e6, f4d4a51dd07544089996f623e2c9ebf3; no voters:
I20260924 00:14:39.005707 9243 raft_consensus.cc:2781] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:39.005762 9243 raft_consensus.cc:482] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:39.005795 9243 raft_consensus.cc:3037] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.006567 9243 raft_consensus.cc:504] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.006704 9243 leader_election.cc:288] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 election: Requested vote from peers 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931), 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165)
I20260924 00:14:39.006860 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895dd3d3b65e41d99658137179e55523" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6"
I20260924 00:14:39.006901 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895dd3d3b65e41d99658137179e55523" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817f7e26786a4c1e9956f59df649bf46"
I20260924 00:14:39.006963 8881 raft_consensus.cc:3037] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.006964 9161 raft_consensus.cc:3037] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.007710 9161 raft_consensus.cc:2454] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 1.
I20260924 00:14:39.007710 8881 raft_consensus.cc:2454] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 1.
I20260924 00:14:39.007926 8952 leader_election.cc:302] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9155954f4ebb4ddc8e2f218d563616e6, f4d4a51dd07544089996f623e2c9ebf3; no voters:
I20260924 00:14:39.008014 9243 raft_consensus.cc:2781] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:39.008157 9243 raft_consensus.cc:686] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 1 LEADER]: Becoming Leader. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Running, Role: LEADER
I20260924 00:14:39.008273 9243 consensus_queue.cc:237] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.008904 8723 catalog_manager.cc:5495] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130). New cstate: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:39.013425 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.011185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.1.44.130:36657)
0924 00:14:39.011237 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.011238 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.011247 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:39.011311 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.011312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.011328 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.011382 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:39.011508 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.011934 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.012676 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.012873 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.012948 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.013128 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.013221 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.013307 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.013330 (+ 23us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.013347 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 00:14:39.014807 8881 raft_consensus.cc:1261] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Refusing update from remote peer f4d4a51dd07544089996f623e2c9ebf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.014869 9161 raft_consensus.cc:1261] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Refusing update from remote peer f4d4a51dd07544089996f623e2c9ebf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.015095 9245 consensus_queue.cc:1035] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.015230 9243 consensus_queue.cc:1035] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.017328 9248 mvcc.cc:205] Tried to move back new op lower bound from 7332695568442589184 to 7332695568418582528. Current Snapshot: MvccSnapshot[applied={T|T < 7332695568442589184}]
I20260924 00:14:39.019371 9250 mvcc.cc:205] Tried to move back new op lower bound from 7332695568442589184 to 7332695568418582528. Current Snapshot: MvccSnapshot[applied={T|T < 7332695568442589184 or (T in {7332695568442589184})}]
I20260924 00:14:39.019806 8724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:39.019891 8724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:39.020565 8724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Sending DeleteTablet for 3 replicas of tablet 895dd3d3b65e41d99658137179e55523
I20260924 00:14:39.020840 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet 895dd3d3b65e41d99658137179e55523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33758
I20260924 00:14:39.020866 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet 895dd3d3b65e41d99658137179e55523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:56238
I20260924 00:14:39.020956 9001 tablet_service.cc:1514] Processing DeleteTablet for tablet 895dd3d3b65e41d99658137179e55523 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41280
I20260924 00:14:39.021015 8724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:39.021145 8724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:39.021173 9259 tablet_replica.cc:330] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: stopping tablet replica
I20260924 00:14:39.021252 9259 raft_consensus.cc:2229] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:39.021330 9259 raft_consensus.cc:2258] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.021708 9259 ts_tablet_manager.cc:1905] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.022275 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 9c0e726655a04219a2d84948ec9e921f (DEFAULT_TABLE table=test-table [id=f20cd3f9d2484e0ea55968a4f8abd9b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.022410 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c0e726655a04219a2d84948ec9e921f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.023254 9259 ts_tablet_manager.cc:1918] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:39.023322 9259 log.cc:1196] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/895dd3d3b65e41d99658137179e55523
I20260924 00:14:39.023582 9259 ts_tablet_manager.cc:1939] T 895dd3d3b65e41d99658137179e55523 P f4d4a51dd07544089996f623e2c9ebf3: Deleting consensus metadata
I20260924 00:14:39.023744 9235 tablet_bootstrap.cc:492] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Bootstrap starting.
I20260924 00:14:39.023900 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 9c0e726655a04219a2d84948ec9e921f (DEFAULT_TABLE table=test-table [id=f20cd3f9d2484e0ea55968a4f8abd9b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.023937 8709 catalog_manager.cc:4841] TS f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657): tablet 895dd3d3b65e41d99658137179e55523 (table test-table [id=41ea1980ace24522841621b37cbb49c0]) successfully deleted
I20260924 00:14:39.023995 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c0e726655a04219a2d84948ec9e921f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.024195 9235 tablet_bootstrap.cc:654] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.024878 9235 tablet_bootstrap.cc:492] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: No bootstrap required, opened a new log
I20260924 00:14:39.024925 9235 ts_tablet_manager.cc:1397] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:39.025060 9235 raft_consensus.cc:348] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.025118 9235 raft_consensus.cc:374] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.025138 9235 raft_consensus.cc:729] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.025192 9235 consensus_queue.cc:260] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.025266 9239 tablet_bootstrap.cc:492] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Bootstrap starting.
I20260924 00:14:39.025297 9235 ts_tablet_manager.cc:1428] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.025533 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 9c0e726655a04219a2d84948ec9e921f (DEFAULT_TABLE table=test-table [id=f20cd3f9d2484e0ea55968a4f8abd9b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.025645 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c0e726655a04219a2d84948ec9e921f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.025662 9239 tablet_bootstrap.cc:654] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.026324 9239 tablet_bootstrap.cc:492] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: No bootstrap required, opened a new log
I20260924 00:14:39.026362 9239 ts_tablet_manager.cc:1397] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:39.026494 9239 raft_consensus.cc:348] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.026548 9239 raft_consensus.cc:374] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.026568 9239 raft_consensus.cc:729] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.026607 9239 consensus_queue.cc:260] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.026624 9236 tablet_bootstrap.cc:492] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Bootstrap starting.
I20260924 00:14:39.026678 9239 ts_tablet_manager.cc:1428] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.027302 9236 tablet_bootstrap.cc:654] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.027973 9236 tablet_bootstrap.cc:492] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: No bootstrap required, opened a new log
I20260924 00:14:39.028035 9236 ts_tablet_manager.cc:1397] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:39.028172 9236 raft_consensus.cc:348] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.028240 9236 raft_consensus.cc:374] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.028272 9236 raft_consensus.cc:729] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9155954f4ebb4ddc8e2f218d563616e6, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.028322 9236 consensus_queue.cc:260] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.028402 9236 ts_tablet_manager.cc:1428] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:39.033285 9257 tablet_replica.cc:330] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: stopping tablet replica
I20260924 00:14:39.033386 9257 raft_consensus.cc:2229] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:39.033481 9257 raft_consensus.cc:2258] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.033629 9258 tablet_replica.cc:330] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: stopping tablet replica
I20260924 00:14:39.033716 9258 raft_consensus.cc:2229] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:39.033799 9258 raft_consensus.cc:2258] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.034235 9258 ts_tablet_manager.cc:1905] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.034242 9257 ts_tablet_manager.cc:1905] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.035604 9258 ts_tablet_manager.cc:1918] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:39.035689 9258 log.cc:1196] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/895dd3d3b65e41d99658137179e55523
I20260924 00:14:39.035701 9257 ts_tablet_manager.cc:1918] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:39.035759 9257 log.cc:1196] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/895dd3d3b65e41d99658137179e55523
I20260924 00:14:39.035969 9258 ts_tablet_manager.cc:1939] T 895dd3d3b65e41d99658137179e55523 P 817f7e26786a4c1e9956f59df649bf46: Deleting consensus metadata
I20260924 00:14:39.036019 9257 ts_tablet_manager.cc:1939] T 895dd3d3b65e41d99658137179e55523 P 9155954f4ebb4ddc8e2f218d563616e6: Deleting consensus metadata
I20260924 00:14:39.036375 8708 catalog_manager.cc:4841] TS 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931): tablet 895dd3d3b65e41d99658137179e55523 (table test-table [id=41ea1980ace24522841621b37cbb49c0]) successfully deleted
I20260924 00:14:39.036417 8709 catalog_manager.cc:4841] TS 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165): tablet 895dd3d3b65e41d99658137179e55523 (table test-table [id=41ea1980ace24522841621b37cbb49c0]) successfully deleted
W20260924 00:14:39.119855 8928 tablet.cc:2369] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:14:39.188406 9208 tablet.cc:2369] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:14:39.268028 9068 tablet.cc:2369] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:14:39.340420 9242 raft_consensus.cc:482] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:39.340507 9242 raft_consensus.cc:504] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.341042 9242 leader_election.cc:288] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165), f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657)
I20260924 00:14:39.344251 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.341440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:35765 (local address 127.1.44.130:36657)
0924 00:14:39.341492 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.341494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.341525 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:14:39.341604 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.341604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.341613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.341663 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:39.341867 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.342439 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.343503 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.343903 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.343988 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.343996 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.344143 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.344180 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 00:14:39.344259 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.341221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36165 (local address 127.1.44.131:53469)
0924 00:14:39.341301 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.341333 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.341410 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.341585 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.341586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.341588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:39.341685 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.341688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.342469 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.342470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.343470 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.343473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.343987 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.343988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.344073 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.344076 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.344147 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.344187 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":44}
I20260924 00:14:39.344267 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.341292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:53469 (local address 127.1.44.129:36165)
0924 00:14:39.341352 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.341354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.341390 (+ 36us) server_negotiation.cc:407] Connection header received
0924 00:14:39.341498 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.341499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.341509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.341568 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:39.341777 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.342380 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.343570 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.343905 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.343989 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.343996 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.344167 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.344201 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 00:14:39.344272 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.341352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:36657 (local address 127.1.44.131:35765)
0924 00:14:39.341466 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.341477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.341539 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.341688 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.341688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.341690 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:39.341777 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.341778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.342532 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.342532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.343416 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.343418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.343986 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.343987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.344059 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.344063 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.344112 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.344168 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260924 00:14:39.344559 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c0e726655a04219a2d84948ec9e921f" candidate_uuid: "9155954f4ebb4ddc8e2f218d563616e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817f7e26786a4c1e9956f59df649bf46" is_pre_election: true
I20260924 00:14:39.344646 8881 raft_consensus.cc:2454] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9155954f4ebb4ddc8e2f218d563616e6 in term 0.
I20260924 00:14:39.344686 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c0e726655a04219a2d84948ec9e921f" candidate_uuid: "9155954f4ebb4ddc8e2f218d563616e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d4a51dd07544089996f623e2c9ebf3" is_pre_election: true
I20260924 00:14:39.344828 9021 raft_consensus.cc:2454] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9155954f4ebb4ddc8e2f218d563616e6 in term 0.
I20260924 00:14:39.344795 9093 leader_election.cc:302] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, 9155954f4ebb4ddc8e2f218d563616e6; no voters:
I20260924 00:14:39.344944 9242 raft_consensus.cc:2781] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:39.344988 9242 raft_consensus.cc:482] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:39.345026 9242 raft_consensus.cc:3037] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.345677 9242 raft_consensus.cc:504] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.345811 9242 leader_election.cc:288] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [CANDIDATE]: Term 1 election: Requested vote from peers 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165), f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657)
I20260924 00:14:39.345952 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c0e726655a04219a2d84948ec9e921f" candidate_uuid: "9155954f4ebb4ddc8e2f218d563616e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817f7e26786a4c1e9956f59df649bf46"
I20260924 00:14:39.346019 8881 raft_consensus.cc:3037] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.346006 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c0e726655a04219a2d84948ec9e921f" candidate_uuid: "9155954f4ebb4ddc8e2f218d563616e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d4a51dd07544089996f623e2c9ebf3"
I20260924 00:14:39.346110 9021 raft_consensus.cc:3037] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.346601 8881 raft_consensus.cc:2454] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9155954f4ebb4ddc8e2f218d563616e6 in term 1.
I20260924 00:14:39.346807 9093 leader_election.cc:302] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, 9155954f4ebb4ddc8e2f218d563616e6; no voters:
I20260924 00:14:39.346843 9021 raft_consensus.cc:2454] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9155954f4ebb4ddc8e2f218d563616e6 in term 1.
I20260924 00:14:39.346902 9242 raft_consensus.cc:2781] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:39.346992 9242 raft_consensus.cc:686] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 LEADER]: Becoming Leader. State: Replica: 9155954f4ebb4ddc8e2f218d563616e6, State: Running, Role: LEADER
I20260924 00:14:39.347066 9242 consensus_queue.cc:237] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.347747 8724 catalog_manager.cc:5495] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131). New cstate: current_term: 1 leader_uuid: "9155954f4ebb4ddc8e2f218d563616e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } health_report { overall_health: HEALTHY } } }
Sep 24 00:14:39 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208878, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.190@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:39.402305 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.395764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44167 (local address 127.0.0.1:37530)
0924 00:14:39.395938 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.395948 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.396033 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.396174 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.396175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.396474 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:39.396524 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.396526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.397159 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.397160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.397667 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.397669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.398030 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.398030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.398080 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.398577 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:39.398581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:39.400026 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:39.401339 (+ 1313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:39.401345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:39.401347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:39.401542 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:39.401711 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:39.401712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:39.401712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:39.401765 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:39.401973 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:39.401979 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:39.402030 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.402085 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.402166 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260924 00:14:39.402307 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.395833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.1.44.190:44167)
0924 00:14:39.395914 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.395916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.396026 (+ 110us) server_negotiation.cc:407] Connection header received
0924 00:14:39.396091 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.396092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.396101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.396170 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:39.396604 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.397073 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.397745 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.397924 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.398031 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.400105 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:39.400107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:39.400107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:39.400113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:39.401269 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:39.401606 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:39.401607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:39.401607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:39.401655 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:39.401816 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:39.401817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:39.401817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:39.401906 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:39.401953 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.402115 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.402209 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":56}
I20260924 00:14:39.404173 8724 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:37530:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 00:14:39.404361 8724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:39.405725 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 904d24384d6c4437b18a039d8f16d68a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b3f20f81d9b4e0980b0a041a5e9e559]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:39.405733 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 904d24384d6c4437b18a039d8f16d68a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b3f20f81d9b4e0980b0a041a5e9e559]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:39.405761 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 904d24384d6c4437b18a039d8f16d68a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b3f20f81d9b4e0980b0a041a5e9e559]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 00:14:39.405871 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 904d24384d6c4437b18a039d8f16d68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.405871 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 904d24384d6c4437b18a039d8f16d68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.405903 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 904d24384d6c4437b18a039d8f16d68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.407281 9236 tablet_bootstrap.cc:492] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: Bootstrap starting.
I20260924 00:14:39.407281 9239 tablet_bootstrap.cc:492] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: Bootstrap starting.
I20260924 00:14:39.407294 9235 tablet_bootstrap.cc:492] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: Bootstrap starting.
I20260924 00:14:39.407680 9235 tablet_bootstrap.cc:654] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.407678 9239 tablet_bootstrap.cc:654] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.407691 9236 tablet_bootstrap.cc:654] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.408236 9239 tablet_bootstrap.cc:492] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: No bootstrap required, opened a new log
I20260924 00:14:39.408289 9239 ts_tablet_manager.cc:1397] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:39.408335 9235 tablet_bootstrap.cc:492] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: No bootstrap required, opened a new log
I20260924 00:14:39.408393 9235 ts_tablet_manager.cc:1397] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:39.408416 9239 raft_consensus.cc:348] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.408485 9239 raft_consensus.cc:374] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.408514 9239 raft_consensus.cc:729] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.408538 9235 raft_consensus.cc:348] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.408561 9239 consensus_queue.cc:260] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.408607 9235 raft_consensus.cc:374] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.408638 9235 raft_consensus.cc:729] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.408649 9239 ts_tablet_manager.cc:1428] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.408687 9235 consensus_queue.cc:260] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.408667 9243 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.408753 9243 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.408800 9235 ts_tablet_manager.cc:1428] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:39.408924 9240 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.408998 9236 tablet_bootstrap.cc:492] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: No bootstrap required, opened a new log
I20260924 00:14:39.409019 9240 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.409036 9236 ts_tablet_manager.cc:1397] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 00:14:39.409173 9236 raft_consensus.cc:348] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.409240 9236 raft_consensus.cc:374] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.409263 9236 raft_consensus.cc:729] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9155954f4ebb4ddc8e2f218d563616e6, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.409307 9236 consensus_queue.cc:260] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.409396 9236 ts_tablet_manager.cc:1428] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.409437 9242 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.409528 9242 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.578398 9243 raft_consensus.cc:482] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:39.578505 9243 raft_consensus.cc:504] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.578666 9243 leader_election.cc:288] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931), 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165)
I20260924 00:14:39.578909 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904d24384d6c4437b18a039d8f16d68a" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6" is_pre_election: true
I20260924 00:14:39.579016 9161 raft_consensus.cc:2454] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 0.
I20260924 00:14:39.578979 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904d24384d6c4437b18a039d8f16d68a" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817f7e26786a4c1e9956f59df649bf46" is_pre_election: true
I20260924 00:14:39.579103 8881 raft_consensus.cc:2454] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 0.
I20260924 00:14:39.579183 8952 leader_election.cc:302] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9155954f4ebb4ddc8e2f218d563616e6, f4d4a51dd07544089996f623e2c9ebf3; no voters:
I20260924 00:14:39.579319 9243 raft_consensus.cc:2781] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:39.579355 9243 raft_consensus.cc:482] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:39.579370 9243 raft_consensus.cc:3037] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.579975 9243 raft_consensus.cc:504] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.580102 9243 leader_election.cc:288] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 election: Requested vote from peers 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931), 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165)
I20260924 00:14:39.580274 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904d24384d6c4437b18a039d8f16d68a" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6"
I20260924 00:14:39.580298 8881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "904d24384d6c4437b18a039d8f16d68a" candidate_uuid: "f4d4a51dd07544089996f623e2c9ebf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "817f7e26786a4c1e9956f59df649bf46"
I20260924 00:14:39.580369 9161 raft_consensus.cc:3037] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.580369 8881 raft_consensus.cc:3037] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.581085 9161 raft_consensus.cc:2454] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 1.
I20260924 00:14:39.581085 8881 raft_consensus.cc:2454] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4d4a51dd07544089996f623e2c9ebf3 in term 1.
I20260924 00:14:39.581270 8953 leader_election.cc:302] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, f4d4a51dd07544089996f623e2c9ebf3; no voters:
I20260924 00:14:39.581365 9243 raft_consensus.cc:2781] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:39.581466 9243 raft_consensus.cc:686] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [term 1 LEADER]: Becoming Leader. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Running, Role: LEADER
I20260924 00:14:39.581507 9243 consensus_queue.cc:237] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } }
I20260924 00:14:39.581768 9275 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: TxnStatusTablet state changed. Reason: New leader f4d4a51dd07544089996f623e2c9ebf3. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.581862 9275 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:39.582002 9277 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 00:14:39.582132 9277 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 00:14:39.582084 8724 catalog_manager.cc:5495] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130). New cstate: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:39.590063 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.587719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.1.44.130:36657)
0924 00:14:39.587761 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.587763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.587775 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:14:39.587855 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.587857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.587876 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.587931 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:39.588041 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.588507 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.589260 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.589469 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.589550 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.589795 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.589841 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.589914 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.589981 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.590000 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:14:39.590101 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.587661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:36657 (local address 127.0.0.1:41296)
0924 00:14:39.587734 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.587747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.587795 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.587938 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.587939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.587940 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:39.587985 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.587988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.588590 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.588591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.589173 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.589174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.589648 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.589649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.589698 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.589710 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.589915 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.589924 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.589977 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.590028 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 00:14:39.591171 8881 raft_consensus.cc:1261] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Refusing update from remote peer f4d4a51dd07544089996f623e2c9ebf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.591178 9161 raft_consensus.cc:1261] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Refusing update from remote peer f4d4a51dd07544089996f623e2c9ebf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.591387 9275 consensus_queue.cc:1035] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.591482 9243 consensus_queue.cc:1035] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.591989 9242 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: TxnStatusTablet state changed. Reason: New leader f4d4a51dd07544089996f623e2c9ebf3. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.591980 9240 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: TxnStatusTablet state changed. Reason: New leader f4d4a51dd07544089996f623e2c9ebf3. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.592062 9242 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.592062 9240 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.592581 9275 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.592669 9275 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:39.593328 9240 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.593421 9240 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P 817f7e26786a4c1e9956f59df649bf46: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.595696 9275 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.595765 9275 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P f4d4a51dd07544089996f623e2c9ebf3: This TxnStatusTablet replica's current role is: LEADER
I20260924 00:14:39.596076 9242 tablet_replica.cc:439] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4d4a51dd07544089996f623e2c9ebf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } } peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } }
I20260924 00:14:39.596156 9242 tablet_replica.cc:442] T 904d24384d6c4437b18a039d8f16d68a P 9155954f4ebb4ddc8e2f218d563616e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 00:14:39.596881 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.594068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33760 (local address 127.1.44.131:34931)
0924 00:14:39.594168 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.594170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.594177 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:39.594224 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.594225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.594243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.594341 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:39.594448 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.594925 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.595930 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.596247 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.596477 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.596514 (+ 37us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.596634 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.596726 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.596778 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.596798 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":59,"threads_started":1}
I20260924 00:14:39.601545 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.599068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:53521 (local address 127.1.44.130:36657)
0924 00:14:39.599108 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.599111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.599236 (+ 125us) server_negotiation.cc:407] Connection header received
0924 00:14:39.599310 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.599311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.599320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.599366 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:39.599495 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.599908 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.600948 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.601115 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.601166 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.601355 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.601378 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.601415 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.601488 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.601502 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:39.601599 9294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.598988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:36657 (local address 127.1.44.131:53521)
0924 00:14:39.599144 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.599156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.599245 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.599386 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.599386 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.599388 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:39.599438 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.599439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.599970 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.599971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.600867 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.600869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.601202 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.601204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.601266 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.601279 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.601434 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.601435 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.601486 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.601527 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260924 00:14:39.603619 8881 raft_consensus.cc:1261] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Refusing update from remote peer 9155954f4ebb4ddc8e2f218d563616e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.603700 9021 raft_consensus.cc:1261] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Refusing update from remote peer 9155954f4ebb4ddc8e2f218d563616e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.603834 9264 consensus_queue.cc:1035] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.603921 9242 consensus_queue.cc:1035] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.605108 9281 mvcc.cc:205] Tried to move back new op lower bound from 7332695570854801408 to 7332695569806127104. Current Snapshot: MvccSnapshot[applied={T|T < 7332695570854801408}]
I20260924 00:14:39.612951 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.610335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34931 (local address 127.1.44.130:39977)
0924 00:14:39.610469 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.610479 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.610542 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.610653 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.610654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.610655 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:39.610686 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.610687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.611540 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.611541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.612194 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.612196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.612627 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.612628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.612662 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.612672 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.612844 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.612845 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.612875 (+ 30us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.612901 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":72,"threads_started":1}
I20260924 00:14:39.612970 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.610410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:39977 (local address 127.1.44.131:34931)
0924 00:14:39.610472 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.610474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.610524 (+ 50us) server_negotiation.cc:407] Connection header received
0924 00:14:39.610583 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.610584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.610593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.610652 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:39.610722 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.611478 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.612265 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.612557 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.612623 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.612737 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.612764 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.612845 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.612878 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.612909 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:14:39.622277 8723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:39.622365 8723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:39.622933 8723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Sending DeleteTablet for 3 replicas of tablet 9c0e726655a04219a2d84948ec9e921f
I20260924 00:14:39.623147 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c0e726655a04219a2d84948ec9e921f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:56238
I20260924 00:14:39.623165 9001 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c0e726655a04219a2d84948ec9e921f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41280
I20260924 00:14:39.623451 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c0e726655a04219a2d84948ec9e921f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33758
I20260924 00:14:39.626192 9304 tablet_replica.cc:330] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: stopping tablet replica
I20260924 00:14:39.626269 9304 raft_consensus.cc:2229] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:39.626356 9304 raft_consensus.cc:2258] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.626607 9304 ts_tablet_manager.cc:1905] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.628360 9304 ts_tablet_manager.cc:1918] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:39.628427 9304 log.cc:1196] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/9c0e726655a04219a2d84948ec9e921f
I20260924 00:14:39.628654 9304 ts_tablet_manager.cc:1939] T 9c0e726655a04219a2d84948ec9e921f P 9155954f4ebb4ddc8e2f218d563616e6: Deleting consensus metadata
I20260924 00:14:39.629328 8708 catalog_manager.cc:4841] TS 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931): tablet 9c0e726655a04219a2d84948ec9e921f (table test-table [id=f20cd3f9d2484e0ea55968a4f8abd9b5]) successfully deleted
I20260924 00:14:39.630147 9302 tablet_replica.cc:330] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: stopping tablet replica
I20260924 00:14:39.630290 9303 tablet_replica.cc:330] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: stopping tablet replica
I20260924 00:14:39.630404 9302 raft_consensus.cc:2229] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:39.630501 9303 raft_consensus.cc:2229] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:39.630584 9302 raft_consensus.cc:2258] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.630661 9303 raft_consensus.cc:2258] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.630981 9302 ts_tablet_manager.cc:1905] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.631304 9303 ts_tablet_manager.cc:1905] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.632907 9302 ts_tablet_manager.cc:1918] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:39.632972 9302 log.cc:1196] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/9c0e726655a04219a2d84948ec9e921f
I20260924 00:14:39.633221 9302 ts_tablet_manager.cc:1939] T 9c0e726655a04219a2d84948ec9e921f P 817f7e26786a4c1e9956f59df649bf46: Deleting consensus metadata
I20260924 00:14:39.633497 8709 catalog_manager.cc:4841] TS 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165): tablet 9c0e726655a04219a2d84948ec9e921f (table test-table [id=f20cd3f9d2484e0ea55968a4f8abd9b5]) successfully deleted
I20260924 00:14:39.634595 9303 ts_tablet_manager.cc:1918] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:39.634723 9303 log.cc:1196] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/9c0e726655a04219a2d84948ec9e921f
I20260924 00:14:39.634977 9303 ts_tablet_manager.cc:1939] T 9c0e726655a04219a2d84948ec9e921f P f4d4a51dd07544089996f623e2c9ebf3: Deleting consensus metadata
I20260924 00:14:39.635286 8709 catalog_manager.cc:4841] TS f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657): tablet 9c0e726655a04219a2d84948ec9e921f (table test-table [id=f20cd3f9d2484e0ea55968a4f8abd9b5]) successfully deleted
Sep 24 00:14:39 dist-test-slave-cm7b krb5kdc[8679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 00:14:39 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:39 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:14:39.655284 8773 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:39.648611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.1.44.190:44167)
0924 00:14:39.648653 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.648655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.648800 (+ 145us) server_negotiation.cc:407] Connection header received
0924 00:14:39.648876 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.648877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.648888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.648941 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:39.649621 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.650397 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.651530 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.651837 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.651907 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.655159 (+ 3252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37542: BlockingRecv error: recv got EOF from 127.0.0.1:37542 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 24 00:14:39 dist-test-slave-cm7b krb5kdc[8679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM
I20260924 00:14:39.667609 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.659222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.1.44.190:44167)
0924 00:14:39.659264 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.659266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.659326 (+ 60us) server_negotiation.cc:407] Connection header received
0924 00:14:39.659407 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.659407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.659416 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.659473 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:39.660077 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.660838 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.661819 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.662119 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.662192 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.664773 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:39.664774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:39.664775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:39.664782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:39.666394 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:39.666776 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:39.666777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:39.666777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:39.666837 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:39.667044 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:39.667045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:39.667045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:39.667170 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:39.667318 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.667446 (+ 128us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.667519 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:39.669266 8723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:14:39.669437 8723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:39.670799 9141 tablet_service.cc:1467] Processing CreateTablet for tablet fa59ab1a772b47cb8d7e71ca1eb622bd (DEFAULT_TABLE table=test-table [id=986bddda69cd483ea255b79e23a89d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.670861 8861 tablet_service.cc:1467] Processing CreateTablet for tablet fa59ab1a772b47cb8d7e71ca1eb622bd (DEFAULT_TABLE table=test-table [id=986bddda69cd483ea255b79e23a89d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.670929 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa59ab1a772b47cb8d7e71ca1eb622bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.670979 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa59ab1a772b47cb8d7e71ca1eb622bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.672324 9235 tablet_bootstrap.cc:492] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Bootstrap starting.
I20260924 00:14:39.672323 9001 tablet_service.cc:1467] Processing CreateTablet for tablet fa59ab1a772b47cb8d7e71ca1eb622bd (DEFAULT_TABLE table=test-table [id=986bddda69cd483ea255b79e23a89d23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.672431 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa59ab1a772b47cb8d7e71ca1eb622bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.672724 9235 tablet_bootstrap.cc:654] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.673568 9235 tablet_bootstrap.cc:492] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: No bootstrap required, opened a new log
I20260924 00:14:39.673609 9235 ts_tablet_manager.cc:1397] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:39.673728 9236 tablet_bootstrap.cc:492] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Bootstrap starting.
I20260924 00:14:39.673736 9235 raft_consensus.cc:348] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.673795 9235 raft_consensus.cc:374] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.673815 9235 raft_consensus.cc:729] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.673857 9235 consensus_queue.cc:260] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.673940 9235 ts_tablet_manager.cc:1428] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.674149 9236 tablet_bootstrap.cc:654] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.674463 9239 tablet_bootstrap.cc:492] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Bootstrap starting.
I20260924 00:14:39.674798 9239 tablet_bootstrap.cc:654] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.675397 9239 tablet_bootstrap.cc:492] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: No bootstrap required, opened a new log
I20260924 00:14:39.675437 9239 ts_tablet_manager.cc:1397] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:39.675529 9239 raft_consensus.cc:348] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.675587 9239 raft_consensus.cc:374] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.675607 9239 raft_consensus.cc:729] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.675649 9239 consensus_queue.cc:260] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.675726 9239 ts_tablet_manager.cc:1428] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.676061 9236 tablet_bootstrap.cc:492] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: No bootstrap required, opened a new log
I20260924 00:14:39.676108 9236 ts_tablet_manager.cc:1397] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 00:14:39.676254 9236 raft_consensus.cc:348] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.676309 9236 raft_consensus.cc:374] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.676330 9236 raft_consensus.cc:729] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9155954f4ebb4ddc8e2f218d563616e6, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.676397 9236 consensus_queue.cc:260] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.676479 9236 ts_tablet_manager.cc:1428] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.913748 9240 raft_consensus.cc:482] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:39.913838 9240 raft_consensus.cc:504] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.914214 9240 leader_election.cc:288] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657), 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931)
I20260924 00:14:39.917304 9289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.914463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:39497 (local address 127.1.44.131:34931)
0924 00:14:39.914516 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.914518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.914543 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:14:39.914643 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.914644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.914654 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.914704 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:39.914861 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.915537 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.916548 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.916976 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.917038 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.917045 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.917181 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.917238 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 00:14:39.917313 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.914379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34931 (local address 127.1.44.129:39497)
0924 00:14:39.914485 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.914496 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.914564 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.914722 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.914723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.914725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:39.914789 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.914791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.915605 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.915606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.916464 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.916467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.917047 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.917048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.917107 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.917111 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.917172 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.917219 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":48,"threads_started":1}
I20260924 00:14:39.917549 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.914439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:45721 (local address 127.1.44.130:36657)
0924 00:14:39.914491 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.914493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.914577 (+ 84us) server_negotiation.cc:407] Connection header received
0924 00:14:39.914660 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.914660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.914667 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.914735 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:14:39.915272 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.915809 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.916746 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.917172 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.917233 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.917240 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.917425 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.917471 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 00:14:39.917557 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa59ab1a772b47cb8d7e71ca1eb622bd" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6" is_pre_election: true
I20260924 00:14:39.917641 9161 raft_consensus.cc:2454] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 0.
I20260924 00:14:39.917811 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.914361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:36657 (local address 127.1.44.129:45721)
0924 00:14:39.914482 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:39.914493 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:39.914568 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:39.914752 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:39.914753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:39.914755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:14:39.915148 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.915151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.915904 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.915905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.916663 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:39.916665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.917245 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.917246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:39.917321 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.917326 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:14:39.917664 (+ 338us) client_negotiation.cc:240] Negotiation successful
0924 00:14:39.917716 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260924 00:14:39.917814 8812 leader_election.cc:302] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, 9155954f4ebb4ddc8e2f218d563616e6; no voters:
I20260924 00:14:39.918030 9240 raft_consensus.cc:2781] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:39.918068 9240 raft_consensus.cc:482] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:39.918087 9240 raft_consensus.cc:3037] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.918108 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa59ab1a772b47cb8d7e71ca1eb622bd" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d4a51dd07544089996f623e2c9ebf3" is_pre_election: true
I20260924 00:14:39.918180 9021 raft_consensus.cc:2454] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 0.
I20260924 00:14:39.918730 9240 raft_consensus.cc:504] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.918864 9240 leader_election.cc:288] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 election: Requested vote from peers f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657), 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931)
I20260924 00:14:39.918970 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa59ab1a772b47cb8d7e71ca1eb622bd" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d4a51dd07544089996f623e2c9ebf3"
I20260924 00:14:39.919014 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa59ab1a772b47cb8d7e71ca1eb622bd" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6"
I20260924 00:14:39.919054 9021 raft_consensus.cc:3037] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.919070 9161 raft_consensus.cc:3037] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:39.919710 9161 raft_consensus.cc:2454] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 1.
I20260924 00:14:39.919812 9021 raft_consensus.cc:2454] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 1.
I20260924 00:14:39.919864 8812 leader_election.cc:302] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, 9155954f4ebb4ddc8e2f218d563616e6; no voters:
I20260924 00:14:39.919947 9240 raft_consensus.cc:2781] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:39.920106 9240 raft_consensus.cc:686] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 1 LEADER]: Becoming Leader. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Running, Role: LEADER
I20260924 00:14:39.920164 9240 consensus_queue.cc:237] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.920841 8723 catalog_manager.cc:5495] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "817f7e26786a4c1e9956f59df649bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:39.959060 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:39.956723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56254 (local address 127.1.44.129:36165)
0924 00:14:39.956869 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:39.956871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:39.956879 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:39.956915 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:39.956916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:39.956934 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:39.957018 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:39.957154 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.957648 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.958337 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:39.958526 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:39.958574 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:39.958704 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:39.958797 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:39.958856 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:39.958889 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 00:14:39.958921 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":85,"threads_started":1}
I20260924 00:14:39.960155 9161 raft_consensus.cc:1261] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Refusing update from remote peer 817f7e26786a4c1e9956f59df649bf46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.960155 9021 raft_consensus.cc:1261] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Refusing update from remote peer 817f7e26786a4c1e9956f59df649bf46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:39.960454 9240 consensus_queue.cc:1035] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.960599 9322 consensus_queue.cc:1035] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:39.963714 8724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:39.963819 8724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:39.964421 8724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Sending DeleteTablet for 3 replicas of tablet fa59ab1a772b47cb8d7e71ca1eb622bd
I20260924 00:14:39.964659 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet fa59ab1a772b47cb8d7e71ca1eb622bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33758
I20260924 00:14:39.964697 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet fa59ab1a772b47cb8d7e71ca1eb622bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:56238
I20260924 00:14:39.964741 9304 tablet_replica.cc:330] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: stopping tablet replica
I20260924 00:14:39.964785 9304 raft_consensus.cc:2229] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:39.964831 9304 raft_consensus.cc:2258] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.964869 9001 tablet_service.cc:1514] Processing DeleteTablet for tablet fa59ab1a772b47cb8d7e71ca1eb622bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:39 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41280
I20260924 00:14:39.964908 9302 tablet_replica.cc:330] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: stopping tablet replica
I20260924 00:14:39.964865 8724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 00:14:39.964957 9302 raft_consensus.cc:2229] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 1 LEADER]: Raft consensus shutting down.
W20260924 00:14:39.964996 8724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:14:39.965001 9303 tablet_replica.cc:330] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: stopping tablet replica
I20260924 00:14:39.965034 9302 raft_consensus.cc:2258] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.965041 9303 raft_consensus.cc:2229] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:39.965080 9303 raft_consensus.cc:2258] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:39.965469 9303 ts_tablet_manager.cc:1905] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.965567 9304 ts_tablet_manager.cc:1905] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.965646 9302 ts_tablet_manager.cc:1905] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:39.967012 9303 ts_tablet_manager.cc:1918] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:39.967012 9304 ts_tablet_manager.cc:1918] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:39.967059 9304 log.cc:1196] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/fa59ab1a772b47cb8d7e71ca1eb622bd
I20260924 00:14:39.967059 9303 log.cc:1196] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/fa59ab1a772b47cb8d7e71ca1eb622bd
I20260924 00:14:39.967082 9302 ts_tablet_manager.cc:1918] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:14:39.967132 9302 log.cc:1196] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/fa59ab1a772b47cb8d7e71ca1eb622bd
I20260924 00:14:39.967306 9304 ts_tablet_manager.cc:1939] T fa59ab1a772b47cb8d7e71ca1eb622bd P 9155954f4ebb4ddc8e2f218d563616e6: Deleting consensus metadata
I20260924 00:14:39.967304 9303 ts_tablet_manager.cc:1939] T fa59ab1a772b47cb8d7e71ca1eb622bd P f4d4a51dd07544089996f623e2c9ebf3: Deleting consensus metadata
I20260924 00:14:39.967378 9302 ts_tablet_manager.cc:1939] T fa59ab1a772b47cb8d7e71ca1eb622bd P 817f7e26786a4c1e9956f59df649bf46: Deleting consensus metadata
I20260924 00:14:39.967650 8709 catalog_manager.cc:4841] TS f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657): tablet fa59ab1a772b47cb8d7e71ca1eb622bd (table test-table [id=986bddda69cd483ea255b79e23a89d23]) successfully deleted
I20260924 00:14:39.967694 8708 catalog_manager.cc:4841] TS 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931): tablet fa59ab1a772b47cb8d7e71ca1eb622bd (table test-table [id=986bddda69cd483ea255b79e23a89d23]) successfully deleted
I20260924 00:14:39.967710 8709 catalog_manager.cc:4841] TS 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165): tablet fa59ab1a772b47cb8d7e71ca1eb622bd (table test-table [id=986bddda69cd483ea255b79e23a89d23]) successfully deleted
I20260924 00:14:39.969249 8861 tablet_service.cc:1467] Processing CreateTablet for tablet 45e39448222c4a25962d211b53c6baab (DEFAULT_TABLE table=test-table [id=6bf6d2ef2008499a9b50f4d33de35bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.969272 9141 tablet_service.cc:1467] Processing CreateTablet for tablet 45e39448222c4a25962d211b53c6baab (DEFAULT_TABLE table=test-table [id=6bf6d2ef2008499a9b50f4d33de35bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.969272 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 45e39448222c4a25962d211b53c6baab (DEFAULT_TABLE table=test-table [id=6bf6d2ef2008499a9b50f4d33de35bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:14:39.969352 8861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e39448222c4a25962d211b53c6baab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.969367 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e39448222c4a25962d211b53c6baab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.969378 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45e39448222c4a25962d211b53c6baab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:39.970481 9236 tablet_bootstrap.cc:492] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Bootstrap starting.
I20260924 00:14:39.970547 9239 tablet_bootstrap.cc:492] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Bootstrap starting.
I20260924 00:14:39.970741 9236 tablet_bootstrap.cc:654] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.970860 9239 tablet_bootstrap.cc:654] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.971086 9235 tablet_bootstrap.cc:492] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46: Bootstrap starting.
I20260924 00:14:39.971271 9236 tablet_bootstrap.cc:492] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: No bootstrap required, opened a new log
I20260924 00:14:39.971311 9236 ts_tablet_manager.cc:1397] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:39.971426 9236 raft_consensus.cc:348] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.971478 9236 raft_consensus.cc:374] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.971493 9236 raft_consensus.cc:729] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9155954f4ebb4ddc8e2f218d563616e6, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.971487 9235 tablet_bootstrap.cc:654] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:39.971549 9236 consensus_queue.cc:260] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.971639 9236 ts_tablet_manager.cc:1428] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.972337 9239 tablet_bootstrap.cc:492] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: No bootstrap required, opened a new log
I20260924 00:14:39.972334 9235 tablet_bootstrap.cc:492] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46: No bootstrap required, opened a new log
I20260924 00:14:39.972376 9239 ts_tablet_manager.cc:1397] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 00:14:39.972379 9235 ts_tablet_manager.cc:1397] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:39.972448 9239 raft_consensus.cc:348] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.972489 9239 raft_consensus.cc:374] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.972502 9239 raft_consensus.cc:729] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d4a51dd07544089996f623e2c9ebf3, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.972518 9235 raft_consensus.cc:348] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.972575 9235 raft_consensus.cc:374] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:39.972556 9239 consensus_queue.cc:260] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.972595 9235 raft_consensus.cc:729] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Initialized, Role: FOLLOWER
I20260924 00:14:39.972643 9239 ts_tablet_manager.cc:1428] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:39.972640 9235 consensus_queue.cc:260] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:39.972719 9235 ts_tablet_manager.cc:1428] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:40.081157 9328 raft_consensus.cc:482] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:14:40.081235 9328 raft_consensus.cc:504] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:40.081413 9328 leader_election.cc:288] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657), 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931)
I20260924 00:14:40.081589 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e39448222c4a25962d211b53c6baab" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6" is_pre_election: true
I20260924 00:14:40.081597 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e39448222c4a25962d211b53c6baab" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d4a51dd07544089996f623e2c9ebf3" is_pre_election: true
I20260924 00:14:40.081682 9161 raft_consensus.cc:2454] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 0.
I20260924 00:14:40.081682 9021 raft_consensus.cc:2454] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 0.
I20260924 00:14:40.081826 8812 leader_election.cc:302] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, 9155954f4ebb4ddc8e2f218d563616e6; no voters:
I20260924 00:14:40.081952 9328 raft_consensus.cc:2781] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:14:40.081991 9328 raft_consensus.cc:482] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:14:40.082012 9328 raft_consensus.cc:3037] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:40.082612 9328 raft_consensus.cc:504] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:40.082751 9328 leader_election.cc:288] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 election: Requested vote from peers f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657), 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931)
I20260924 00:14:40.082878 9021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e39448222c4a25962d211b53c6baab" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d4a51dd07544089996f623e2c9ebf3"
I20260924 00:14:40.082922 9161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45e39448222c4a25962d211b53c6baab" candidate_uuid: "817f7e26786a4c1e9956f59df649bf46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9155954f4ebb4ddc8e2f218d563616e6"
I20260924 00:14:40.082981 9161 raft_consensus.cc:3037] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:40.082989 9021 raft_consensus.cc:3037] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:40.083623 9161 raft_consensus.cc:2454] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 1.
I20260924 00:14:40.083711 9021 raft_consensus.cc:2454] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 817f7e26786a4c1e9956f59df649bf46 in term 1.
I20260924 00:14:40.083799 8812 leader_election.cc:302] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 817f7e26786a4c1e9956f59df649bf46, 9155954f4ebb4ddc8e2f218d563616e6; no voters:
I20260924 00:14:40.083928 9328 raft_consensus.cc:2781] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:40.083997 9328 raft_consensus.cc:686] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 1 LEADER]: Becoming Leader. State: Replica: 817f7e26786a4c1e9956f59df649bf46, State: Running, Role: LEADER
I20260924 00:14:40.084043 9328 consensus_queue.cc:237] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } }
I20260924 00:14:40.084599 8724 catalog_manager.cc:5495] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "817f7e26786a4c1e9956f59df649bf46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "817f7e26786a4c1e9956f59df649bf46" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 } health_report { overall_health: UNKNOWN } } }
I20260924 00:14:40.093863 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:40.091431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:43327 (local address 127.1.44.130:36657)
0924 00:14:40.091470 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:40.091472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:40.091571 (+ 99us) server_negotiation.cc:407] Connection header received
0924 00:14:40.091631 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:40.091632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:40.091639 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:40.091708 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:40.091853 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.092329 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.093179 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.093376 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.093459 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:40.093622 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:40.093659 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:40.093716 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:40.093776 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 00:14:40.093796 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:14:40.093899 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:40.091386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:36657 (local address 127.1.44.129:43327)
0924 00:14:40.091517 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:40.091528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:40.091592 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:40.091711 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:40.091712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:40.091714 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:40.091759 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:40.091760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.092405 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.092406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:40.093101 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:40.093102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.093458 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.093459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:40.093512 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:40.093525 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:40.093718 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:40.093720 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:40.093771 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:14:40.093814 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260924 00:14:40.095719 9021 raft_consensus.cc:1261] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Refusing update from remote peer 817f7e26786a4c1e9956f59df649bf46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:40.095767 9161 raft_consensus.cc:1261] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Refusing update from remote peer 817f7e26786a4c1e9956f59df649bf46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:14:40.095932 9328 consensus_queue.cc:1035] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d4a51dd07544089996f623e2c9ebf3" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:40.096030 9322 consensus_queue.cc:1035] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9155954f4ebb4ddc8e2f218d563616e6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:14:40.101989 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:40.099888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48095 (local address 127.1.44.129:36165)
0924 00:14:40.099920 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:40.099922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:40.099968 (+ 46us) server_negotiation.cc:407] Connection header received
0924 00:14:40.100025 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:40.100026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:40.100035 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:40.100094 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:14:40.100193 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.100644 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.101353 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.101559 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.101639 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:40.101773 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:14:40.101806 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:14:40.101849 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:40.101912 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 00:14:40.101931 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:40.102023 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:40.099853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36165 (local address 127.1.44.130:48095)
0924 00:14:40.099928 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:40.099938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:40.099979 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:40.100097 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:40.100097 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:40.100098 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 00:14:40.100126 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:40.100127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.100730 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.100730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:40.101259 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:40.101260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:40.101639 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:40.101640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:40.101692 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:40.101697 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 00:14:40.101867 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 00:14:40.101868 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 00:14:40.101911 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 00:14:40.101955 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 00:14:40.107810 8724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:40.107916 8724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:14:40.108421 8724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 753e6d1a74db4d0bb52d0fe267d66f30: Sending DeleteTablet for 3 replicas of tablet 45e39448222c4a25962d211b53c6baab
I20260924 00:14:40.108666 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e39448222c4a25962d211b53c6baab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:40 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33758
I20260924 00:14:40.108759 9001 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e39448222c4a25962d211b53c6baab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:40 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41280
I20260924 00:14:40.108794 9304 tablet_replica.cc:330] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: stopping tablet replica
I20260924 00:14:40.108850 9303 tablet_replica.cc:330] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: stopping tablet replica
I20260924 00:14:40.108860 9304 raft_consensus.cc:2229] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:40.108892 9304 raft_consensus.cc:2258] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:40.108907 9303 raft_consensus.cc:2229] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:14:40.108942 9303 raft_consensus.cc:2258] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:40.109216 9303 ts_tablet_manager.cc:1905] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:14:40.109216 8861 tablet_service.cc:1514] Processing DeleteTablet for tablet 45e39448222c4a25962d211b53c6baab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:14:40 UTC) from {username='oryx', principal='oryx/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:56238
I20260924 00:14:40.109294 9302 tablet_replica.cc:330] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46: stopping tablet replica
I20260924 00:14:40.109318 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 8796
I20260924 00:14:40.109337 9302 raft_consensus.cc:2229] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:14:40.109381 9302 raft_consensus.cc:2258] T 45e39448222c4a25962d211b53c6baab P 817f7e26786a4c1e9956f59df649bf46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:14:40.110951 9303 ts_tablet_manager.cc:1918] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:40.111001 9303 log.cc:1196] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/45e39448222c4a25962d211b53c6baab
I20260924 00:14:40.111228 9303 ts_tablet_manager.cc:1939] T 45e39448222c4a25962d211b53c6baab P f4d4a51dd07544089996f623e2c9ebf3: Deleting consensus metadata
I20260924 00:14:40.112699 8709 catalog_manager.cc:4841] TS f4d4a51dd07544089996f623e2c9ebf3 (127.1.44.130:36657): tablet 45e39448222c4a25962d211b53c6baab (table test-table [id=6bf6d2ef2008499a9b50f4d33de35bc7]) successfully deleted
I20260924 00:14:40.115450 9304 ts_tablet_manager.cc:1905] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 00:14:40.116367 8709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:36165 (error 108)
I20260924 00:14:40.116617 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 8936
I20260924 00:14:40.116780 9304 ts_tablet_manager.cc:1918] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 00:14:40.116833 9304 log.cc:1196] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/45e39448222c4a25962d211b53c6baab
I20260924 00:14:40.117013 9304 ts_tablet_manager.cc:1939] T 45e39448222c4a25962d211b53c6baab P 9155954f4ebb4ddc8e2f218d563616e6: Deleting consensus metadata
I20260924 00:14:40.117376 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:40.116734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36165 (local address 127.0.0.1:56258)
0924 00:14:40.117287 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:40.117330 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:36165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":34,"threads_started":1}
W20260924 00:14:40.117460 8709 catalog_manager.cc:4584] TS 817f7e26786a4c1e9956f59df649bf46 (127.1.44.129:36165): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45e39448222c4a25962d211b53c6baab: Network error: Client connection negotiation failed: client connection to 127.1.44.129:36165: connect: Connection refused (error 111)
I20260924 00:14:40.117506 8708 catalog_manager.cc:4841] TS 9155954f4ebb4ddc8e2f218d563616e6 (127.1.44.131:34931): tablet 45e39448222c4a25962d211b53c6baab (table test-table [id=6bf6d2ef2008499a9b50f4d33de35bc7]) successfully deleted
I20260924 00:14:40.123569 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9076
I20260924 00:14:40.128940 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 8692
2026-09-24T00:14:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2937 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:40 dist-test-slave-cm7b krb5kdc[9343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:40 dist-test-slave-cm7b krb5kdc[9343](info): set up 2 sockets
Sep 24 00:14:40 dist-test-slave-cm7b krb5kdc[9343](info): commencing operation
krb5kdc: starting...
W20260924 00:14:41.185290 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:41 dist-test-slave-cm7b krb5kdc[9343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:41.301427 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:38543
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:38543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:41.379297 9356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:41.379470 9356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:41.379492 9356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:41.380618 9356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:41.380657 9356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:41.380672 9356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:41.380684 9356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:41.380735 9356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:41.382155 9356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:38543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:38543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:41.382390 9356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:41.382601 9356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:41.387306 9362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:41.387367 9356 server_base.cc:1034] running on GCE node
W20260924 00:14:41.387328 9361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:41.387306 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:41.387746 9356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:41.388015 9356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:41.389174 9356 hybrid_clock.cc:648] HybridClock initialized: now 1790208881389161 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T00:14:41Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1268 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:41 dist-test-slave-cm7b krb5kdc[9369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:41 dist-test-slave-cm7b krb5kdc[9369](info): set up 2 sockets
Sep 24 00:14:41 dist-test-slave-cm7b krb5kdc[9369](info): commencing operation
krb5kdc: starting...
W20260924 00:14:42.423781 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:14:42 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:14:42.534091 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44659
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:42.610641 9382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:42.610873 9382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:42.610912 9382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:42.612052 9382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:42.612120 9382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:42.612146 9382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:42.612179 9382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:42.612202 9382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:42.613601 9382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:42.613874 9382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:42.614084 9382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:42.618649 9390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:42.618634 9387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:42.618634 9388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:42.618875 9382 server_base.cc:1034] running on GCE node
I20260924 00:14:42.619149 9382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:42.619485 9382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:42.620666 9382 hybrid_clock.cc:648] HybridClock initialized: now 1790208882620634 us; error 44 us; skew 500 ppm
Sep 24 00:14:42 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:42.629050 9382 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:42.629491 9382 webserver.cc:460] Webserver started at http://127.1.44.190:41581/ using document root <none> and password file <none>
I20260924 00:14:42.629706 9382 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:42.629773 9382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:42.629910 9382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:42.630888 9382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "24543389ac63416fa7686b9243ec48ad"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "8a1acfe2db7b1a5a1b0c8cf67f926ce9"
server_key_iv: "655214b46deba288a35f8880697867d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.631287 9382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "24543389ac63416fa7686b9243ec48ad"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "8a1acfe2db7b1a5a1b0c8cf67f926ce9"
server_key_iv: "655214b46deba288a35f8880697867d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.632547 9382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:42.633261 9397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.633481 9382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:42.633592 9382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "24543389ac63416fa7686b9243ec48ad"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "8a1acfe2db7b1a5a1b0c8cf67f926ce9"
server_key_iv: "655214b46deba288a35f8880697867d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.633679 9382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:42.660468 9382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:42.662657 9382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:42.662817 9382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:42.666767 9382 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44659
I20260924 00:14:42.666903 9449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44659 every 8 connection(s)
I20260924 00:14:42.667161 9382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:42.667950 9450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:42.668392 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9382
I20260924 00:14:42.668479 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:42.668679 1202 external_mini_cluster.cc:1428] Setting key a030e5c8f15130703126a6dc55b846c3
I20260924 00:14:42.670826 9450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Bootstrap starting.
I20260924 00:14:42.671490 9450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:42.671782 9450 log.cc:824] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:42.672478 9450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: No bootstrap required, opened a new log
I20260924 00:14:42.674886 9450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:42.675050 9450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:42.675089 9450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24543389ac63416fa7686b9243ec48ad, State: Initialized, Role: FOLLOWER
I20260924 00:14:42.675244 9450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:42.675350 9450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:42.675395 9450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:42.675443 9450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 0 FOLLOWER]: Advancing to term 1
Sep 24 00:14:42 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:42.675999 9450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:42.676112 9450 leader_election.cc:302] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24543389ac63416fa7686b9243ec48ad; no voters:
I20260924 00:14:42.676283 9450 leader_election.cc:288] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:42.676321 9455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:42.676508 9455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 LEADER]: Becoming Leader. State: Replica: 24543389ac63416fa7686b9243ec48ad, State: Running, Role: LEADER
I20260924 00:14:42.676539 9450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:42.676749 9455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:42.677376 9457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24543389ac63416fa7686b9243ec48ad. Latest consensus state: current_term: 1 leader_uuid: "24543389ac63416fa7686b9243ec48ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } } }
I20260924 00:14:42.677464 9457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:42.678249 9456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24543389ac63416fa7686b9243ec48ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } } }
I20260924 00:14:42.678634 9456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: This master's current role is: LEADER
W20260924 00:14:42.678661 9471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:14:42.678704 9471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:14:42.678792 9464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:42.678992 9464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:42.679949 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:42.669207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.1.44.190:44659)
0924 00:14:42.669396 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:42.669405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:42.669438 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:14:42.669936 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:42.669944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:42.670000 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:42.670146 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:42.670715 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:42.672100 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:42.673133 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:42.673522 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:42.673599 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:42.676180 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:42.676184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:42.676184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:42.676197 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:42.678283 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:42.678713 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:42.678716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:42.678716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:42.678817 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:42.679034 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:42.679035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:42.679035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:42.679356 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:42.679490 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:42.679628 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 00:14:42.679752 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260924 00:14:42.680910 9464 catalog_manager.cc:1347] Generated new cluster ID: 158e06bccc174fc08dbe84c09f62f325
I20260924 00:14:42.680961 9464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:42.698448 9464 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:42.699141 9464 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:42.705049 9464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Generated new TSK 0
I20260924 00:14:42.705202 9464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:14:42.730073 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44659
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:42.807893 9476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:42.808072 9476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:42.808095 9476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:42.809242 9476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:42.809284 9476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:42.809335 9476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:42.810712 9476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:42.810938 9476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:42.811161 9476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:42.815932 9481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:42.815975 9484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:42.815950 9482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:42.815968 9476 server_base.cc:1034] running on GCE node
I20260924 00:14:42.816242 9476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:42.816490 9476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:42.817655 9476 hybrid_clock.cc:648] HybridClock initialized: now 1790208882817640 us; error 26 us; skew 500 ppm
Sep 24 00:14:42 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:42.820116 9476 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:14:42.820536 9476 webserver.cc:460] Webserver started at http://127.1.44.129:39215/ using document root <none> and password file <none>
I20260924 00:14:42.820727 9476 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:42.820767 9476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:42.820849 9476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:42.821731 9476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "5b804d5c1a3a4d02afbc7a288e6c07c3"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "4186e05428b70e841079b4e21b5e617d"
server_key_iv: "7408dac5d3efe928f6b6ff3d0a02d41b"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.822017 9476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "5b804d5c1a3a4d02afbc7a288e6c07c3"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "4186e05428b70e841079b4e21b5e617d"
server_key_iv: "7408dac5d3efe928f6b6ff3d0a02d41b"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.823233 9476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:42.823925 9491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.824098 9476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:42.824193 9476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "5b804d5c1a3a4d02afbc7a288e6c07c3"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "4186e05428b70e841079b4e21b5e617d"
server_key_iv: "7408dac5d3efe928f6b6ff3d0a02d41b"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.824270 9476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:42.849517 9476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:42.851557 9476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:42.851706 9476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:42.851908 9476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:42.852185 9476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:42.852239 9476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.852279 9476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:42.852310 9476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.857892 9476 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:42689
I20260924 00:14:42.857934 9604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:42689 every 8 connection(s)
I20260924 00:14:42.858254 9476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:14:42 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:42.864300 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9476
I20260924 00:14:42.864408 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:42.864612 1202 external_mini_cluster.cc:1428] Setting key 6bacca7e029d24ae3a539ec831744b57
I20260924 00:14:42.867128 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:42.858934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:59741 (local address 127.1.44.190:44659)
0924 00:14:42.858982 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:42.858984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:42.859171 (+ 187us) server_negotiation.cc:407] Connection header received
0924 00:14:42.859689 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:42.859690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:42.859701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:42.859762 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:42.860656 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:42.861153 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:42.862132 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:42.862335 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:42.862413 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:42.864426 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:42.864427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:42.864427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:42.864434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:42.865811 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:42.866212 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:42.866213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:42.866214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:42.866272 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:42.866516 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:42.866517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:42.866518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:42.866626 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:42.866702 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:42.866964 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 00:14:42.867039 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:14:42.867312 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:42.858889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:59741)
0924 00:14:42.859054 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:42.859091 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:42.859582 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:42.859833 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:42.859837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:42.860375 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:42.860558 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:42.860562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:42.861246 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:42.861248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:42.862041 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:42.862043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:42.862421 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:42.862422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:42.862543 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:42.863026 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:42.863034 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:42.864305 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:42.865922 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:42.865929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:42.865931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:42.866138 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:42.866339 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:42.866340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:42.866341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:42.866449 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:42.866693 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:42.866699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:42.866830 (+ 131us) client_negotiation.cc:769] Sending connection context
0924 00:14:42.866931 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:14:42.867075 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260924 00:14:42.867735 9605 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44659
I20260924 00:14:42.867830 9605 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:42.868002 9605 heartbeater.cc:511] Master 127.1.44.190:44659 requested a full tablet report, sending...
I20260924 00:14:42.868494 9414 ts_manager.cc:194] Registered new tserver with Master: 5b804d5c1a3a4d02afbc7a288e6c07c3 (127.1.44.129:42689)
I20260924 00:14:42.869457 9414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:59741
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:14:42.887876 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44659
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:42.967077 9610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:42.967273 9610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:42.967295 9610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:42.968434 9610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:42.968474 9610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:42.968500 9610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:42.969883 9610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:42.970116 9610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:42.970325 9610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:42.974926 9618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:42.974934 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:42.975037 9610 server_base.cc:1034] running on GCE node
W20260924 00:14:42.974934 9615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:42.975360 9610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:42.975543 9610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:42.976696 9610 hybrid_clock.cc:648] HybridClock initialized: now 1790208882976672 us; error 38 us; skew 500 ppm
Sep 24 00:14:42 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:42.979341 9610 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:14:42.979686 9610 webserver.cc:460] Webserver started at http://127.1.44.130:45713/ using document root <none> and password file <none>
I20260924 00:14:42.979849 9610 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:42.979887 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:42.979967 9610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:42.980842 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "386b3ece58704c81b21e1c215387d435"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "fc709f31d8dbeee663919da871095734"
server_key_iv: "b84b44f2522a118562504e06dd559483"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.981115 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "386b3ece58704c81b21e1c215387d435"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "fc709f31d8dbeee663919da871095734"
server_key_iv: "b84b44f2522a118562504e06dd559483"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.982246 9610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:42.982911 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.983131 9610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:42.983238 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "386b3ece58704c81b21e1c215387d435"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "fc709f31d8dbeee663919da871095734"
server_key_iv: "b84b44f2522a118562504e06dd559483"
server_key_version: "encryptionkey@0"
I20260924 00:14:42.983328 9610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:42.991539 9610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:42.993180 9610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:42.993326 9610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:42.993520 9610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:42.993791 9610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:42.993824 9610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.993844 9610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:42.993889 9610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:42.999362 9610 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:40235
I20260924 00:14:42.999406 9738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:40235 every 8 connection(s)
I20260924 00:14:42.999673 9610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:43.001703 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9610
I20260924 00:14:43.001866 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:43.002063 1202 external_mini_cluster.cc:1428] Setting key d65ab51bf2f1c4cc49bbb7825b237d1e
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:43.009009 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.000455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:39355 (local address 127.1.44.190:44659)
0924 00:14:43.000500 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.000502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.000726 (+ 224us) server_negotiation.cc:407] Connection header received
0924 00:14:43.001131 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.001133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.001142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.001192 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.002141 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.002768 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.003819 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.004029 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.004115 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.006160 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:43.006161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:43.006162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:43.006169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:43.007655 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:43.008064 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:43.008065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:43.008066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:43.008123 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:43.008369 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:43.008370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:43.008370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:43.008481 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:43.008557 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:43.008847 (+ 290us) server_negotiation.cc:299] Negotiation successful
0924 00:14:43.008921 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:14:43.009202 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.000453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:39355)
0924 00:14:43.000614 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.000646 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.001055 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.001252 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.001254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.001772 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.002047 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.002051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.002853 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.002854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.003727 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.003730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.004123 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.004124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.004251 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.004716 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.004724 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.006068 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:43.007753 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:43.007759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:43.007761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:43.007968 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:43.008192 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:43.008193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:43.008193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:43.008300 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:43.008561 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:43.008567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:43.008692 (+ 125us) client_negotiation.cc:769] Sending connection context
0924 00:14:43.008822 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 00:14:43.008967 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
I20260924 00:14:43.009648 9739 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44659
I20260924 00:14:43.009732 9739 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:43.009917 9739 heartbeater.cc:511] Master 127.1.44.190:44659 requested a full tablet report, sending...
I20260924 00:14:43.010272 9414 ts_manager.cc:194] Registered new tserver with Master: 386b3ece58704c81b21e1c215387d435 (127.1.44.130:40235)
I20260924 00:14:43.010715 9414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:39355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:14:43.026185 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44659
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:43.103138 9744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:43.103394 9744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:43.103418 9744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:43.104554 9744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:43.104594 9744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:43.104620 9744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:43.105988 9744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:43.106194 9744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:43.106426 9744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:43.110975 9749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.110996 9750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.110998 9752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.111389 9744 server_base.cc:1034] running on GCE node
I20260924 00:14:43.111586 9744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:43.111764 9744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:43.112901 9744 hybrid_clock.cc:648] HybridClock initialized: now 1790208883112856 us; error 62 us; skew 500 ppm
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:43.115132 9744 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:14:43.115583 9744 webserver.cc:460] Webserver started at http://127.1.44.131:43017/ using document root <none> and password file <none>
I20260924 00:14:43.115806 9744 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:43.115878 9744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:43.116016 9744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:43.116991 9744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "9dbba44438884db1a1afde93ff6051ea"
format_stamp: "Formatted at 2026-09-24 00:14:43 on dist-test-slave-cm7b"
server_key: "cfb377cad515d1649746c1b2a98a15a5"
server_key_iv: "3103220505df615c78b86a197da7f9df"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.117336 9744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "9dbba44438884db1a1afde93ff6051ea"
format_stamp: "Formatted at 2026-09-24 00:14:43 on dist-test-slave-cm7b"
server_key: "cfb377cad515d1649746c1b2a98a15a5"
server_key_iv: "3103220505df615c78b86a197da7f9df"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.118579 9744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:14:43.119352 9759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.119498 9744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.119577 9744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "9dbba44438884db1a1afde93ff6051ea"
format_stamp: "Formatted at 2026-09-24 00:14:43 on dist-test-slave-cm7b"
server_key: "cfb377cad515d1649746c1b2a98a15a5"
server_key_iv: "3103220505df615c78b86a197da7f9df"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.119626 9744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:43.155005 9744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:43.157202 9744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:43.157371 9744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:43.157603 9744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:43.157903 9744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:43.157958 9744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.157999 9744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:43.158026 9744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.163693 9744 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:32965
I20260924 00:14:43.163726 9872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:32965 every 8 connection(s)
I20260924 00:14:43.164075 9744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:14:43.170780 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9744
I20260924 00:14:43.170902 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:43.171128 1202 external_mini_cluster.cc:1428] Setting key e5995de0ff3ffb4ebd6ceb9883a03f8f
I20260924 00:14:43.173422 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.164752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:59329 (local address 127.1.44.190:44659)
0924 00:14:43.164793 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.164796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.164984 (+ 188us) server_negotiation.cc:407] Connection header received
0924 00:14:43.165507 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.165508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.165522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.165575 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.166483 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.166983 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.168065 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.168278 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.168363 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.170390 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:43.170392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:43.170392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:43.170400 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:43.171989 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:43.172480 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:43.172481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:43.172482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:43.172540 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:43.172788 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:43.172789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:43.172790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:43.172901 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:43.172975 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:43.173237 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 00:14:43.173311 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:14:43.173616 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.164716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:59329)
0924 00:14:43.164856 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.164898 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.165393 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.165651 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.165656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.166203 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.166379 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.166384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.167091 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.167092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.167947 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.167950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.168371 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.168372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.168496 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.168995 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.169008 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.170293 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:43.172093 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:43.172099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:43.172101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:43.172391 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:43.172606 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:14:43.172607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:14:43.172607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:14:43.172719 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:14:43.172967 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:43.172973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:43.173108 (+ 135us) client_negotiation.cc:769] Sending connection context
0924 00:14:43.173216 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 00:14:43.173365 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260924 00:14:43.174202 9873 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44659
I20260924 00:14:43.174294 9873 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:43.174460 9873 heartbeater.cc:511] Master 127.1.44.190:44659 requested a full tablet report, sending...
I20260924 00:14:43.174804 9414 ts_manager.cc:194] Registered new tserver with Master: 9dbba44438884db1a1afde93ff6051ea (127.1.44.131:32965)
I20260924 00:14:43.175132 9414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:59329
I20260924 00:14:43.182850 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab.
Entry for principal oryx/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab.
WARNING: no policy specified for oryx/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab.
Entry for principal oryx/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab.
WARNING: no policy specified for oryx/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab.
Entry for principal oryx/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab.
I20260924 00:14:43.247843 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9476
I20260924 00:14:43.252621 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9610
I20260924 00:14:43.256858 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9744
I20260924 00:14:43.261034 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9382
I20260924 00:14:43.266034 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44659
--webserver_interface=127.1.44.190
--webserver_port=41581
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:14:43.343912 9882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:43.344125 9882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:43.344158 9882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:43.345260 9882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:43.345325 9882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:43.345350 9882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:43.345371 9882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:43.345399 9882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:43.346777 9882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=41581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:43.347035 9882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:43.347258 9882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:43.351789 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.351828 9888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.351794 9890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.352046 9882 server_base.cc:1034] running on GCE node
I20260924 00:14:43.352272 9882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:43.352538 9882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:43.353710 9882 hybrid_clock.cc:648] HybridClock initialized: now 1790208883353697 us; error 25 us; skew 500 ppm
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:43.356141 9882 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:14:43.356530 9882 webserver.cc:460] Webserver started at http://127.1.44.190:41581/ using document root <none> and password file <none>
I20260924 00:14:43.356739 9882 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:43.356823 9882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:43.358047 9882 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:43.358659 9897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.358878 9882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:43.358978 9882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "24543389ac63416fa7686b9243ec48ad"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "8a1acfe2db7b1a5a1b0c8cf67f926ce9"
server_key_iv: "655214b46deba288a35f8880697867d2"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.359328 9882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:43.376827 9882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:43.378484 9882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:43.378635 9882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:43.382305 9882 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44659
I20260924 00:14:43.382345 9949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44659 every 8 connection(s)
I20260924 00:14:43.382683 9882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:43.384052 9950 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:14:43.384672 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Bootstrap starting.
I20260924 00:14:43.390163 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9882
I20260924 00:14:43.390441 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:42689
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:44659
--webserver_port=39215
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:14:43.392469 9950 log.cc:824] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:43.393955 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:14:43.394230 9950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Bootstrap complete.
I20260924 00:14:43.396410 9950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:43.396641 9950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24543389ac63416fa7686b9243ec48ad, State: Initialized, Role: FOLLOWER
I20260924 00:14:43.396835 9950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:43.396929 9950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:43.396981 9950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:43.397027 9950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:14:43.397944 9950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:43.398062 9950 leader_election.cc:302] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24543389ac63416fa7686b9243ec48ad; no voters:
I20260924 00:14:43.398253 9950 leader_election.cc:288] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:14:43.398289 9954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:14:43.398401 9954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [term 2 LEADER]: Becoming Leader. State: Replica: 24543389ac63416fa7686b9243ec48ad, State: Running, Role: LEADER
I20260924 00:14:43.398485 9954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } }
I20260924 00:14:43.398514 9950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:43.398767 9956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24543389ac63416fa7686b9243ec48ad. Latest consensus state: current_term: 2 leader_uuid: "24543389ac63416fa7686b9243ec48ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } } }
I20260924 00:14:43.398859 9956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:43.398921 9955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "24543389ac63416fa7686b9243ec48ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24543389ac63416fa7686b9243ec48ad" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44659 } } }
I20260924 00:14:43.399042 9955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:43.399139 9958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:43.400744 9958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:43.400906 9958 catalog_manager.cc:1259] Loaded cluster ID: 158e06bccc174fc08dbe84c09f62f325
I20260924 00:14:43.400957 9958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:43.401733 9958 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:43.401912 9958 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 24543389ac63416fa7686b9243ec48ad: Loaded TSK: 0
I20260924 00:14:43.402125 9958 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:14:43.475453 9952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:43.475658 9952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:43.475692 9952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:43.476806 9952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:43.476878 9952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:43.476920 9952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:43.478296 9952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.129.keytab
--principal=oryx/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=39215
--tserver_master_addrs=127.1.44.190:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:43.478643 9952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:43.478891 9952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:43.484588 9977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.484723 9976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.484795 9952 server_base.cc:1034] running on GCE node
W20260924 00:14:43.484917 9979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.485103 9952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:43.485340 9952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:43.486510 9952 hybrid_clock.cc:648] HybridClock initialized: now 1790208883486492 us; error 31 us; skew 500 ppm
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208883, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:43.489322 9952 init.cc:377] Logged in from keytab as oryx/127.1.44.129@KRBTEST.COM (short username oryx)
I20260924 00:14:43.489760 9952 webserver.cc:460] Webserver started at http://127.1.44.129:39215/ using document root <none> and password file <none>
I20260924 00:14:43.489997 9952 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:43.490051 9952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:43.491400 9952 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:14:43.492013 9986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.492190 9952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:43.492275 9952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "5b804d5c1a3a4d02afbc7a288e6c07c3"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "4186e05428b70e841079b4e21b5e617d"
server_key_iv: "7408dac5d3efe928f6b6ff3d0a02d41b"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.492590 9952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:43.520932 9952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:43.522991 9952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:43.523149 9952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:43.523407 9952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:43.523699 9952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:43.523751 9952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.523785 9952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:43.523815 9952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.529263 9952 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:42689
I20260924 00:14:43.529343 10099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:42689 every 8 connection(s)
I20260924 00:14:43.529666 9952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:43.535384 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 9952
I20260924 00:14:43.535687 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:40235
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:44659
--webserver_port=45713
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.537918 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.530353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:34579)
0924 00:14:43.530545 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.530596 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.531099 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.531558 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.531562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.532119 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.532311 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.532316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.533385 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.533386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.534150 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.534153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.534743 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.534744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.534876 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.535414 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.535422 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.537788 (+ 2366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
W20260924 00:14:43.538241 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.530401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:34579 (local address 127.1.44.190:44659)
0924 00:14:43.530573 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.530582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.530707 (+ 125us) server_negotiation.cc:407] Connection header received
0924 00:14:43.531325 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.531334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.531379 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.531520 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.532431 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.533272 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.534243 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.534640 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.534775 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.537941 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:34579: BlockingRecv error: recv got EOF from 127.1.44.129:34579 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
W20260924 00:14:43.538430 10100 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:44659 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.44.190:44659: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.543313 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.538631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:35519)
0924 00:14:43.538693 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.538705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.538750 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.538916 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.538916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.539020 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.539059 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.539060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.540162 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.540163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.540694 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.540696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.541208 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.541209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.541257 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.541661 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.541665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.543265 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260924 00:14:43.543665 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.538720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:35519 (local address 127.1.44.190:44659)
0924 00:14:43.538798 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.538801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.538811 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:43.538832 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.538833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.538845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.538910 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.539233 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.540035 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.540788 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.541125 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.541198 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.543366 (+ 2168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:35519: BlockingRecv error: recv got EOF from 127.1.44.129:35519 (error 108)
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":58}
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.548092 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.543554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:40211)
0924 00:14:43.543798 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.543808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.543861 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.543993 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.543994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.544147 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.544181 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.544183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.544747 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.544748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.545641 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.545644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.545964 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.545965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.546028 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.546342 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.546346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.548021 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230}
W20260924 00:14:43.548144 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.543601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:40211 (local address 127.1.44.190:44659)
0924 00:14:43.543754 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.543770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.543849 (+ 79us) server_negotiation.cc:407] Connection header received
0924 00:14:43.543912 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.543912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.543920 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.543967 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.544255 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.544670 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.545731 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.545896 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.545940 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.548081 (+ 2141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:40211: BlockingRecv error: recv got EOF from 127.1.44.129:40211 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
W20260924 00:14:43.548406 10100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:14:43.616843 10104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:43.617024 10104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:43.617046 10104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:43.618218 10104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:43.618263 10104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:43.618289 10104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:43.619728 10104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.130.keytab
--principal=oryx/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:40235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=45713
--tserver_master_addrs=127.1.44.190:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:43.619957 10104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:43.620203 10104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:43.625419 10111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.625823 10113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.625872 10110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.627139 10104 server_base.cc:1034] running on GCE node
I20260924 00:14:43.627405 10104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:43.627650 10104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:43.628834 10104 hybrid_clock.cc:648] HybridClock initialized: now 1790208883628816 us; error 30 us; skew 500 ppm
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208883, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:43.631333 10104 init.cc:377] Logged in from keytab as oryx/127.1.44.130@KRBTEST.COM (short username oryx)
I20260924 00:14:43.631693 10104 webserver.cc:460] Webserver started at http://127.1.44.130:45713/ using document root <none> and password file <none>
I20260924 00:14:43.631897 10104 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:43.631947 10104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:43.633219 10104 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:43.633785 10120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.633935 10104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:43.634027 10104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "386b3ece58704c81b21e1c215387d435"
format_stamp: "Formatted at 2026-09-24 00:14:42 on dist-test-slave-cm7b"
server_key: "fc709f31d8dbeee663919da871095734"
server_key_iv: "b84b44f2522a118562504e06dd559483"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.634307 10104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:43.644874 10104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:43.646502 10104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:43.646603 10104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:43.646757 10104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:43.647042 10104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:43.647075 10104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.647094 10104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:43.647143 10104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.652786 10104 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:40235
I20260924 00:14:43.652853 10233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:40235 every 8 connection(s)
I20260924 00:14:43.653167 10104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.659826 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.653833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:38773)
0924 00:14:43.654029 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.654061 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.654463 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.654714 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.654717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.655307 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.655480 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.655484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.656130 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.656131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.656877 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.656879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.657190 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.657191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.657280 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.657672 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.657679 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.659702 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
W20260924 00:14:43.659830 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.653834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:38773 (local address 127.1.44.190:44659)
0924 00:14:43.653897 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.653899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.654139 (+ 240us) server_negotiation.cc:407] Connection header received
0924 00:14:43.654572 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.654574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.654586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.654643 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.655576 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.656052 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.656951 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.657123 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.657171 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.659788 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:38773: BlockingRecv error: recv got EOF from 127.1.44.130:38773 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260924 00:14:43.660012 10234 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:44659 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.44.190:44659: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
I20260924 00:14:43.660784 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10104
I20260924 00:14:43.661072 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:32965
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:44659
--webserver_port=43017
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.664644 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.660221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:32987)
0924 00:14:43.660268 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.660278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.660312 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.660448 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.660448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.660555 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.660592 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.660593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.661474 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.661475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.661943 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.661945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.662390 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.662391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.662490 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.662854 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.662860 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.664603 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 00:14:43.664788 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.660288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:32987 (local address 127.1.44.190:44659)
0924 00:14:43.660337 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.660338 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.660347 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:43.660380 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.660380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.660387 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.660435 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.660661 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.661363 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.662013 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.662303 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.662376 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.664710 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:32987: BlockingRecv error: recv got EOF from 127.1.44.130:32987 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.669060 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.665251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:37825)
0924 00:14:43.665287 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.665295 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.665326 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.665465 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.665466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.665590 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.665613 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.665614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.666197 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.666198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.666668 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.666670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.667010 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.667012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.667062 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.667469 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.667474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.668982 (+ 1508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 00:14:43.669108 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.665313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:37825 (local address 127.1.44.190:44659)
0924 00:14:43.665352 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.665354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.665362 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:43.665382 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.665382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.665390 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.665462 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.665693 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.666123 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.666757 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.666937 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.667007 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.669047 (+ 2040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:37825: BlockingRecv error: recv got EOF from 127.1.44.130:37825 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 00:14:43.669412 10234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:14:43.741988 10237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:43.742173 10237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:43.742197 10237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:43.743312 10237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:43.743350 10237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:43.743382 10237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:43.744701 10237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/krb5kdc/oryx_127.1.44.131.keytab
--principal=oryx/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:32965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=43017
--tserver_master_addrs=127.1.44.190:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:43.744863 10237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:43.745023 10237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:43.749788 10242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.749804 10237 server_base.cc:1034] running on GCE node
W20260924 00:14:43.749771 10243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:43.749761 10245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:43.750332 10237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:43.750537 10237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:43.751701 10237 hybrid_clock.cc:648] HybridClock initialized: now 1790208883751692 us; error 39 us; skew 500 ppm
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208883, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:14:43.754082 10237 init.cc:377] Logged in from keytab as oryx/127.1.44.131@KRBTEST.COM (short username oryx)
I20260924 00:14:43.754431 10237 webserver.cc:460] Webserver started at http://127.1.44.131:43017/ using document root <none> and password file <none>
I20260924 00:14:43.754603 10237 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:43.754653 10237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:43.755919 10237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:14:43.756417 10252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.756609 10237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:43.756691 10237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "9dbba44438884db1a1afde93ff6051ea"
format_stamp: "Formatted at 2026-09-24 00:14:43 on dist-test-slave-cm7b"
server_key: "cfb377cad515d1649746c1b2a98a15a5"
server_key_iv: "3103220505df615c78b86a197da7f9df"
server_key_version: "encryptionkey@0"
I20260924 00:14:43.756953 10237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:43.779826 10237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:43.781729 10237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:43.781915 10237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:43.782131 10237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:43.782421 10237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:43.782476 10237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.782517 10237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:43.782546 10237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:43.788065 10237 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:32965
I20260924 00:14:43.788133 10365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:32965 every 8 connection(s)
I20260924 00:14:43.788461 10237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 00:14:43.795022 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.789135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:42305 (local address 127.1.44.190:44659)
0924 00:14:43.789187 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.789189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.789375 (+ 186us) server_negotiation.cc:407] Connection header received
0924 00:14:43.789787 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.789788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.789797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.789851 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.790656 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.791127 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.792074 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.792252 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.792305 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.794967 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:42305: BlockingRecv error: recv got EOF from 127.1.44.131:42305 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:14:43.795028 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.789097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:42305)
0924 00:14:43.789248 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.789286 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.789708 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.789897 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.789900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.790413 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.790583 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.790587 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.791236 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.791237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.791996 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.791998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.792319 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.792320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.792412 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.792785 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.792792 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.794897 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":93,"threads_started":1}
W20260924 00:14:43.795276 10366 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:44659 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.44.190:44659: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
I20260924 00:14:43.796104 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10237
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.799808 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.795511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:53259)
0924 00:14:43.795561 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.795571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.795610 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.795787 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.795787 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.795906 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.795991 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.795993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.796655 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.796656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.797135 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.797137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.797482 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.797483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.797554 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.797932 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.797937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.799736 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 00:14:43.799932 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.795654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:53259 (local address 127.1.44.190:44659)
0924 00:14:43.795694 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.795696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.795702 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:14:43.795718 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.795719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.795725 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.795770 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.796065 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.796569 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.797214 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.797398 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.797482 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.799860 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:53259: BlockingRecv error: recv got EOF from 127.1.44.131:53259 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:43.803886 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.796531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.1.44.190:44659)
0924 00:14:43.796587 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.796589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.796729 (+ 140us) server_negotiation.cc:407] Connection header received
0924 00:14:43.796799 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.796800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.796810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.796890 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.797464 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.797920 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.798739 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.798934 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.799014 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.800431 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:43.800435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:43.800435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:14:43.800446 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:43.802332 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:43.802762 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:43.802765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:43.802765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:43.802858 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:14:43.803081 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:14:43.803082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:14:43.803083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:14:43.803425 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:43.803560 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:43.803663 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 00:14:43.803784 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:43 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:43.805063 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:43.800591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:54707)
0924 00:14:43.800696 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:43.800705 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:43.800740 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:43.800885 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:43.800885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:43.801019 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:43.801051 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.801052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.801742 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.801743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.802521 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:43.802523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.802913 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.802915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:43.802970 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.803351 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:43.803356 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:43.805024 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260924 00:14:43.805171 10366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:14:43.805181 10103 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:43.800651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:54707 (local address 127.1.44.190:44659)
0924 00:14:43.800717 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:43.800719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:43.800732 (+ 13us) server_negotiation.cc:407] Connection header received
0924 00:14:43.800806 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:43.800807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:43.800816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:43.800866 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:43.801123 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.801667 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.802591 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:43.802846 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:43.802892 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:43.805114 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:54707: BlockingRecv error: recv got EOF from 127.1.44.131:54707 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 24 00:14:44 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:44 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:44.555040 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:44.548739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:58651)
0924 00:14:44.548940 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:44.548956 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:44.549003 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:44.549320 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:44.549321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:44.549468 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:44.549512 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:44.549514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.550485 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.550486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:44.551370 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:44.551373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.551949 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.551951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:44.552017 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:44.552460 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:44.552466 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:44.554950 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":139,"threads_started":1}
W20260924 00:14:44.555154 10371 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:44.549031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:58651 (local address 127.1.44.190:44659)
0924 00:14:44.549141 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:44.549142 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:44.549150 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:44.549188 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:44.549188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:44.549200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:44.549276 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:44.549632 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.550415 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.551485 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.551862 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.551928 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:44.555070 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:58651: BlockingRecv error: recv got EOF from 127.1.44.129:58651 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":59,"threads_started":1}
Sep 24 00:14:44 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:44 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:44.674172 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:44.669830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:52377)
0924 00:14:44.669936 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:44.669948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:44.670008 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:44.670188 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:44.670189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:44.670352 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:44.670395 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:44.670397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.671043 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.671044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:44.671604 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:44.671606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.671968 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.671969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:44.672034 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:44.672456 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:44.672461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:44.674112 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":59,"threads_started":1}
W20260924 00:14:44.674268 10371 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:44.669873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:52377 (local address 127.1.44.190:44659)
0924 00:14:44.669956 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:44.669958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:44.670000 (+ 42us) server_negotiation.cc:407] Connection header received
0924 00:14:44.670096 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:44.670097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:44.670110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:44.670182 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:44.670473 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.670955 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.671692 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.671887 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.671960 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:44.674202 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:52377: BlockingRecv error: recv got EOF from 127.1.44.130:52377 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 24 00:14:44 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:44 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:44.810154 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:44.805520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:43331)
0924 00:14:44.805682 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:44.805694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:44.805761 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:44.805926 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:44.805927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:44.806101 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:44.806142 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:44.806144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.806788 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.806789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:44.807359 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:44.807361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.807723 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.807724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:44.807778 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:44.808232 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:44.808236 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:44.810056 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
W20260924 00:14:44.810186 10371 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:44.805591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:43331 (local address 127.1.44.190:44659)
0924 00:14:44.805673 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:44.805675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:44.805743 (+ 68us) server_negotiation.cc:407] Connection header received
0924 00:14:44.805815 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:44.805816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:44.805847 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:44.805919 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:44.806221 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.806707 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.807444 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:44.807638 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:44.807722 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:44.810124 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:43331: BlockingRecv error: recv got EOF from 127.1.44.131:43331 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 00:14:45 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:45 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:45.559917 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:45.555443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:50807)
0924 00:14:45.555584 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:45.555594 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:45.555677 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:45.555862 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:45.555863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:45.556011 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:45.556057 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:45.556059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.556680 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.556681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:45.557212 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:45.557214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.557523 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.557525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:45.557566 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:45.558024 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:45.558028 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:45.559850 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260924 00:14:45.560047 10375 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:45.555532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:50807 (local address 127.1.44.190:44659)
0924 00:14:45.555644 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:45.555645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:45.555677 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:14:45.555760 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:45.555761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:45.555771 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:45.555849 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:45.556120 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.556609 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.557279 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.557460 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.557510 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:45.559934 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:50807: BlockingRecv error: recv got EOF from 127.1.44.129:50807 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":45,"threads_started":1}
Sep 24 00:14:45 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:45 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:45.679024 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:45.674646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:41373)
0924 00:14:45.674774 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:45.674797 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:45.674857 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:45.675022 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:45.675023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:45.675182 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:45.675293 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:45.675300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.675939 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.675940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:45.676467 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:45.676469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.676786 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.676787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:45.676825 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:45.677231 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:45.677235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:45.678923 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
W20260924 00:14:45.679080 10375 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:45.674700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:41373 (local address 127.1.44.190:44659)
0924 00:14:45.674762 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:45.674764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:45.674862 (+ 98us) server_negotiation.cc:407] Connection header received
0924 00:14:45.674909 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:45.674909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:45.674919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:45.674960 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:45.675367 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.675869 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.676536 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.676711 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.676760 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:45.679014 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:41373: BlockingRecv error: recv got EOF from 127.1.44.130:41373 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 00:14:45 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:45 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:45.815276 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:45.810631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:50673)
0924 00:14:45.810765 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:45.810777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:45.810834 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:45.810991 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:45.810992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:45.811167 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:45.811261 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:45.811263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.811923 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.811924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:45.812458 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:45.812460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.812837 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.812838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:45.812897 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:45.813399 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:45.813404 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:45.815160 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260924 00:14:45.815429 10375 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:45.810771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:50673 (local address 127.1.44.190:44659)
0924 00:14:45.810818 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:45.810820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:45.810830 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:45.810890 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:45.810891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:45.810903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:45.810979 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:45.811347 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.811838 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.812547 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:45.812735 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:45.812837 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:45.815360 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:50673: BlockingRecv error: recv got EOF from 127.1.44.131:50673 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 24 00:14:46 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:46 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:46.565050 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:46.560395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:41531)
0924 00:14:46.560541 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:46.560551 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:46.560605 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:46.560722 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:46.560723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:46.560879 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:46.560927 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:46.560928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.561721 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.561722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:46.562251 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:46.562253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.562662 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.562663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:46.562709 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:46.563179 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:46.563184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:46.564956 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
W20260924 00:14:46.565110 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:46.560481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:41531 (local address 127.1.44.190:44659)
0924 00:14:46.560578 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:46.560580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:46.560593 (+ 13us) server_negotiation.cc:407] Connection header received
0924 00:14:46.560651 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:46.560652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:46.560661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:46.560717 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:46.560988 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.561567 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.562369 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.562594 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.562647 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:46.565028 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:41531: BlockingRecv error: recv got EOF from 127.1.44.129:41531 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":51,"threads_started":1}
Sep 24 00:14:46 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:46 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:46.684046 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:46.679481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:46437)
0924 00:14:46.679603 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:46.679615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:46.679677 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:46.679835 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:46.679836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:46.680013 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:46.680059 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:46.680061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.680724 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.680725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:46.681270 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:46.681272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.681652 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.681653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:46.681710 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:46.682174 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:46.682178 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:46.683964 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
W20260924 00:14:46.684113 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:46.679613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:46437 (local address 127.1.44.190:44659)
0924 00:14:46.679677 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:46.679679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:46.679688 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:46.679736 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:46.679737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:46.679749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:46.679813 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:46.680139 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.680635 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.681361 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.681564 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.681654 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:46.684066 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:46437: BlockingRecv error: recv got EOF from 127.1.44.130:46437 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 00:14:46 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:46 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:46.820680 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:46.815750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:56505)
0924 00:14:46.815912 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:46.815921 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:46.815973 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:46.816148 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:46.816148 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:46.816460 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:46.816552 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:46.816555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.817207 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.817208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:46.817809 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:46.817811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.818178 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.818179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:46.818232 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:46.818796 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:46.818801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:46.820573 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
W20260924 00:14:46.820716 10379 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:46.815850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:56505 (local address 127.1.44.190:44659)
0924 00:14:46.815924 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:46.815926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:46.815963 (+ 37us) server_negotiation.cc:407] Connection header received
0924 00:14:46.816021 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:46.816022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:46.816038 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:46.816138 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:46.816627 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.817122 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.817904 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:46.818104 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:46.818181 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:46.820652 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:56505: BlockingRecv error: recv got EOF from 127.1.44.131:56505 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 24 00:14:47 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:47 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:47.570338 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:47.565566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:46779)
0924 00:14:47.565717 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:47.565727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:47.565785 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:47.565916 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:47.565917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:47.566074 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:47.566122 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:47.566124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.567073 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.567074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:47.567615 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:47.567617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.567923 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.567924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:47.567960 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:47.568581 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:47.568586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:47.570281 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
W20260924 00:14:47.570461 10383 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:47.565654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:46779 (local address 127.1.44.190:44659)
0924 00:14:47.565773 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:47.565774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:47.565784 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:47.565832 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:47.565832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:47.565842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:47.565903 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:47.566210 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.567004 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.567680 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.567863 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.567912 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:47.570371 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:46779: BlockingRecv error: recv got EOF from 127.1.44.129:46779 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
Sep 24 00:14:47 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:47 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:47.692960 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:47.684581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:50059)
0924 00:14:47.684739 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:47.684754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:47.684828 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:47.685061 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:47.685062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:47.685238 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:47.685282 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:47.685284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.686057 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.686058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:47.686571 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:47.686572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.686893 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.686894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:47.686955 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:47.691094 (+ 4139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:47.691099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:47.692900 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
W20260924 00:14:47.693076 10383 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:47.684698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:50059 (local address 127.1.44.190:44659)
0924 00:14:47.684774 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:47.684776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:47.684811 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:14:47.684941 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:47.684943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:47.684956 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:47.685051 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:47.685400 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.685991 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.686643 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.686828 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.686889 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:47.693001 (+ 6112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:50059: BlockingRecv error: recv got EOF from 127.1.44.130:50059 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 24 00:14:47 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:47 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:47.825907 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:47.821131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:49341)
0924 00:14:47.821331 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:47.821343 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:47.821439 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:47.821603 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:47.821604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:47.821792 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:47.821836 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:47.821838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.822517 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.822519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:47.823065 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:47.823066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.823485 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.823486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:47.823542 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:47.824052 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:47.824057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:47.825811 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
W20260924 00:14:47.826043 10383 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:47.821231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:49341 (local address 127.1.44.190:44659)
0924 00:14:47.821311 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:47.821313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:47.821420 (+ 107us) server_negotiation.cc:407] Connection header received
0924 00:14:47.821511 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:47.821512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:47.821532 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:47.821596 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:47.821921 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.822417 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.823149 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:47.823408 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:47.823486 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:47.825959 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:49341: BlockingRecv error: recv got EOF from 127.1.44.131:49341 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 24 00:14:48 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:48 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:48.575291 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:48.570818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:54239)
0924 00:14:48.571024 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:48.571037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:48.571109 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:48.571286 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:48.571287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:48.571454 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:48.571503 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:48.571505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.572132 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.572133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:48.572647 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:48.572649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.572966 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.572966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:48.573012 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:48.573483 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:48.573489 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:48.575177 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":88,"threads_started":1}
W20260924 00:14:48.575383 10387 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:48.570894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:54239 (local address 127.1.44.190:44659)
0924 00:14:48.570999 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:48.571001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:48.571122 (+ 121us) server_negotiation.cc:407] Connection header received
0924 00:14:48.571169 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:48.571170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:48.571184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:48.571268 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:48.571568 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.572058 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.572723 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.572904 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.572954 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:48.575312 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:54239: BlockingRecv error: recv got EOF from 127.1.44.129:54239 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
Sep 24 00:14:48 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:48 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:48.698164 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:48.693495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:42817)
0924 00:14:48.693650 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:48.693666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:48.693746 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:48.693914 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:48.693916 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:48.694090 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:48.694136 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:48.694137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.694805 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.694806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:48.695423 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:48.695425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.695843 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.695844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:48.695908 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:48.696338 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:48.696342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:48.698066 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":58,"threads_started":1}
W20260924 00:14:48.698217 10387 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:48.693653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:42817 (local address 127.1.44.190:44659)
0924 00:14:48.693717 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:48.693719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:48.693729 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:48.693822 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:48.693823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:48.693836 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:48.693911 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:48.694223 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.694712 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.695514 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.695761 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.695835 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:48.698155 (+ 2320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:42817: BlockingRecv error: recv got EOF from 127.1.44.130:42817 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 24 00:14:48 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:48 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:48.831511 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:48.826548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:44397)
0924 00:14:48.826719 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:48.826730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:48.826804 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:48.826980 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:48.826981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:48.827139 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:48.827187 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:48.827188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.828078 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.828079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:48.828758 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:48.828760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.829103 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.829103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:48.829143 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:48.829630 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:48.829635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:48.831440 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":66,"threads_started":1}
W20260924 00:14:48.831568 10387 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:48.826639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44397 (local address 127.1.44.190:44659)
0924 00:14:48.826711 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:48.826714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:48.826792 (+ 78us) server_negotiation.cc:407] Connection header received
0924 00:14:48.826858 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:48.826859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:48.826871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:48.826915 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:48.827362 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.827961 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.828831 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:48.829023 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:48.829096 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:48.831519 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:44397: BlockingRecv error: recv got EOF from 127.1.44.131:44397 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 00:14:49 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:49 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:49.580000 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:49.575739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:56487)
0924 00:14:49.575880 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:49.575894 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:49.575952 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:49.576084 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:49.576085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:49.576234 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:49.576283 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:49.576284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.576898 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.576898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:49.577423 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:49.577425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.577730 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.577730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:49.577775 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:49.578195 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:49.578199 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:49.579944 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
W20260924 00:14:49.580119 10391 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:49.575802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:56487 (local address 127.1.44.190:44659)
0924 00:14:49.575937 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:49.575939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:49.575946 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:14:49.576012 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:49.576013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:49.576022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:49.576073 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:49.576341 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.576831 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.577487 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.577671 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.577718 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:49.580035 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:56487: BlockingRecv error: recv got EOF from 127.1.44.129:56487 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
Sep 24 00:14:49 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:49 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:49.704489 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:49.698658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:54693)
0924 00:14:49.698821 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:49.698832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:49.698907 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:49.699095 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:49.699096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:49.699348 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:49.699411 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:49.699414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.700350 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.700351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:49.701139 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:49.701142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.701582 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.701583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:49.701644 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:49.702137 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:49.702142 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:49.704405 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
W20260924 00:14:49.704777 10391 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:49.698710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:54693 (local address 127.1.44.190:44659)
0924 00:14:49.698762 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:49.698764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:49.698922 (+ 158us) server_negotiation.cc:407] Connection header received
0924 00:14:49.698969 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:49.698970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:49.698999 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:49.699067 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:49.699518 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.700266 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.701237 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.701492 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.701561 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:49.704507 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:54693: BlockingRecv error: recv got EOF from 127.1.44.130:54693 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 24 00:14:49 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:49 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:49.836742 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:49.832054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:45547)
0924 00:14:49.832205 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:49.832224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:49.832281 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:49.832418 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:49.832419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:49.832569 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:49.832614 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:49.832616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.833256 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.833256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:49.834001 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:49.834004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.834363 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.834363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:49.834402 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:49.834817 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:49.834822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:49.836655 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
W20260924 00:14:49.836870 10391 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:49.832169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:45547 (local address 127.1.44.190:44659)
0924 00:14:49.832236 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:49.832238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:49.832263 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:14:49.832339 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:49.832339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:49.832352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:49.832396 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:49.832685 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.833179 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.834080 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:49.834300 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:49.834354 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:49.836783 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:45547: BlockingRecv error: recv got EOF from 127.1.44.131:45547 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 00:14:50 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:50 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:50.585284 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:50.580516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:47383)
0924 00:14:50.580668 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:50.580688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:50.580736 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:50.580889 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:50.580890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:50.581054 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:50.581103 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:50.581104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.581995 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.581996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:50.582523 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:50.582525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.582835 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.582836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:50.582873 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:50.583473 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:50.583478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:50.585216 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
W20260924 00:14:50.585451 10395 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:50.580602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:47383 (local address 127.1.44.190:44659)
0924 00:14:50.580719 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:50.580721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:50.580730 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:50.580789 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:50.580789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:50.580799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:50.580870 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:50.581163 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.581897 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.582589 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.582771 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.582820 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:50.585360 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:47383: BlockingRecv error: recv got EOF from 127.1.44.129:47383 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
Sep 24 00:14:50 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:50 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:50.709753 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:50.705072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:35791)
0924 00:14:50.705254 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:50.705267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:50.705361 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:50.705519 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:50.705520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:50.705691 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:50.705736 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:50.705738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.706412 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.706413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:50.706937 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:50.706939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.707368 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.707369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:50.707430 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:50.707846 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:50.707850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:50.709680 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
W20260924 00:14:50.709846 10395 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:50.705172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:35791 (local address 127.1.44.190:44659)
0924 00:14:50.705258 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:50.705261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:50.705342 (+ 81us) server_negotiation.cc:407] Connection header received
0924 00:14:50.705447 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:50.705448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:50.705460 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:50.705502 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:50.705819 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.706327 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.707022 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.707285 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.707368 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:50.709780 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:35791: BlockingRecv error: recv got EOF from 127.1.44.130:35791 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 24 00:14:50 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:50 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:50.842047 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:50.837327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:35553)
0924 00:14:50.837473 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:50.837490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:50.837545 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:50.837683 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:50.837684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:50.837890 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:50.837935 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:50.837937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.838567 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.838568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:50.839088 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:50.839089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.839693 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.839695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:50.839757 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:50.840239 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:50.840243 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:50.841954 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
W20260924 00:14:50.842087 10395 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:50.837380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:35553 (local address 127.1.44.190:44659)
0924 00:14:50.837442 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:50.837444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:50.837566 (+ 122us) server_negotiation.cc:407] Connection header received
0924 00:14:50.837591 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:50.837592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:50.837606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:50.837669 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:50.838003 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.838490 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.839344 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:50.839617 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:50.839686 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:50.842027 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:35553: BlockingRecv error: recv got EOF from 127.1.44.131:35553 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 24 00:14:51 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:51 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:51.590273 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:51.585783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:33331)
0924 00:14:51.585898 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:51.585916 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:51.585967 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:51.586113 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:51.586114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:51.586272 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:51.586319 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:51.586320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.586957 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.586958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:51.587567 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:51.587570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.587930 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.587931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:51.587979 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:51.588423 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:51.588428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:51.590214 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":78,"threads_started":1}
W20260924 00:14:51.590391 10399 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:51.585852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:33331 (local address 127.1.44.190:44659)
0924 00:14:51.585987 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:51.585989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:51.585998 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:51.586025 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:51.586026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:51.586036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:51.586092 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:51.586386 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.586881 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.587648 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.587841 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.587899 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:51.590308 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:33331: BlockingRecv error: recv got EOF from 127.1.44.129:33331 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
Sep 24 00:14:51 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:51 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:51.714707 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:51.710257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:56131)
0924 00:14:51.710366 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:51.710376 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:51.710430 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:51.710613 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:51.710613 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:51.710762 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:51.710808 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:51.710809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.711617 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.711618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:51.712137 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:51.712138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.712440 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.712441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:51.712477 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:51.712902 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:51.712906 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:51.714615 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":62,"threads_started":1}
W20260924 00:14:51.714743 10399 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:51.710324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:56131 (local address 127.1.44.190:44659)
0924 00:14:51.710491 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:51.710494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:51.710503 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:51.710532 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:51.710533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:51.710543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:51.710598 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:51.710872 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.711551 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.712206 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.712380 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.712430 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:51.714685 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:56131: BlockingRecv error: recv got EOF from 127.1.44.130:56131 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Sep 24 00:14:51 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:51 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:51.847095 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:51.842529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:44127)
0924 00:14:51.842690 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:51.842701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:51.842769 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:51.842895 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:51.842895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:51.843051 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:51.843115 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:51.843116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.843857 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.843859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:51.844401 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:51.844402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.844771 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.844772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:51.844833 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:51.845313 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:51.845331 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:51.847026 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
W20260924 00:14:51.847158 10399 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:51.842645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44127 (local address 127.1.44.190:44659)
0924 00:14:51.842713 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:51.842715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:51.842763 (+ 48us) server_negotiation.cc:407] Connection header received
0924 00:14:51.842811 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:51.842812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:51.842824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:51.842872 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:51.843217 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.843775 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.844495 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:51.844687 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:51.844771 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:51.847111 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:44127: BlockingRecv error: recv got EOF from 127.1.44.131:44127 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 24 00:14:52 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:52 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:52.595180 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:52.590771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:54565)
0924 00:14:52.590947 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:52.590958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:52.591008 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:52.591125 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:52.591125 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:52.591302 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:52.591349 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:52.591350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.592002 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.592003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:52.592536 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:52.592537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.592848 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.592849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:52.592885 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:52.593280 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:52.593284 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:52.595079 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":106,"threads_started":1}
W20260924 00:14:52.595237 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:52.590868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:54565 (local address 127.1.44.190:44659)
0924 00:14:52.591000 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:52.591001 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:52.591010 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:52.591053 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:52.591053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:52.591063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:52.591120 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:52.591434 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.591930 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.592605 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.592785 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.592845 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:52.595150 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:54565: BlockingRecv error: recv got EOF from 127.1.44.129:54565 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
Sep 24 00:14:52 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:52 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:52.723502 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:52.715345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:36215)
0924 00:14:52.715477 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:52.715519 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:52.715596 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:52.715768 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:52.715769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:52.715950 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:52.715993 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:52.715995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.716659 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.716661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:52.717434 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:52.717437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.717803 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.717803 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:52.717862 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:52.721485 (+ 3623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:52.721491 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:52.723432 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
W20260924 00:14:52.723660 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:52.715465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:36215 (local address 127.1.44.190:44659)
0924 00:14:52.715548 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:52.715550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:52.715577 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:14:52.715661 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:52.715662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:52.715675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:52.715744 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:52.716073 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.716575 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.717515 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.717719 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.717803 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:52.723579 (+ 5776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:36215: BlockingRecv error: recv got EOF from 127.1.44.130:36215 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 24 00:14:52 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:52 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:52.852394 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:52.847643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:36457)
0924 00:14:52.847847 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:52.847858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:52.847931 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:52.848083 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:52.848084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:52.848284 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:52.848329 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:52.848331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.848985 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.848986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:52.849541 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:52.849542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.849905 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.849906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:52.849962 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:52.850465 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:52.850470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:52.852290 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
W20260924 00:14:52.852414 10403 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:52.847764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:36457 (local address 127.1.44.190:44659)
0924 00:14:52.847839 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:52.847842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:52.847911 (+ 69us) server_negotiation.cc:407] Connection header received
0924 00:14:52.847985 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:52.847986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:52.848000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:52.848070 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:52.848411 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.848901 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.849631 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:52.849828 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:52.849906 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:52.852361 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:36457: BlockingRecv error: recv got EOF from 127.1.44.131:36457 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 24 00:14:53 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:53 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:53.600311 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:53.595839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:41845)
0924 00:14:53.596000 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:53.596010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:53.596066 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:53.596179 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:53.596180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:53.596335 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:53.596381 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:53.596383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.597006 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.597006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:53.597524 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:53.597525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.597834 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.597835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:53.597871 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:53.598301 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:53.598306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:53.600255 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
W20260924 00:14:53.600488 10407 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:53.595907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:41845 (local address 127.1.44.190:44659)
0924 00:14:53.596012 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:53.596013 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:53.596068 (+ 55us) server_negotiation.cc:407] Connection header received
0924 00:14:53.596118 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:53.596118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:53.596128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:53.596176 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:53.596440 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.596934 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.597590 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.597773 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.597820 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:53.600404 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:41845: BlockingRecv error: recv got EOF from 127.1.44.129:41845 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":46,"threads_started":1}
Sep 24 00:14:53 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:53 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:53.728775 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:53.723942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:42877)
0924 00:14:53.724077 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:53.724088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:53.724149 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:53.724288 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:53.724289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:53.724518 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:53.724594 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:53.724596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.725595 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.725596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:53.726127 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:53.726130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.726492 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.726494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:53.726542 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:53.727045 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:53.727051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:53.728719 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
W20260924 00:14:53.728865 10407 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:53.723999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:42877 (local address 127.1.44.190:44659)
0924 00:14:53.724045 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:53.724048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:53.724133 (+ 85us) server_negotiation.cc:407] Connection header received
0924 00:14:53.724211 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:53.724212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:53.724225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:53.724283 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:53.724688 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.725517 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.726205 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.726391 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.726451 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:53.728800 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:42877: BlockingRecv error: recv got EOF from 127.1.44.130:42877 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 00:14:53 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:53 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:53.857343 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:53.852897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:48719)
0924 00:14:53.853050 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:53.853068 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:53.853116 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:53.853243 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:53.853244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:53.853407 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:53.853451 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:53.853452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.854098 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.854099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:53.854628 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:53.854630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.854986 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.854986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:53.855024 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:53.855498 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:53.855503 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:53.857287 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260924 00:14:53.857450 10407 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:53.853046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:48719 (local address 127.1.44.190:44659)
0924 00:14:53.853089 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:53.853093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:53.853107 (+ 14us) server_negotiation.cc:407] Connection header received
0924 00:14:53.853159 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:53.853160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:53.853171 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:53.853222 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:53.853518 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.853999 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.854717 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:53.854903 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:53.854965 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:53.857378 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:48719: BlockingRecv error: recv got EOF from 127.1.44.131:48719 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 00:14:54 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:54 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:54.606720 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:54.600789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:38167)
0924 00:14:54.600898 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:54.600911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:54.600968 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:54.601193 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:54.601201 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:54.601451 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:54.601507 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:54.601509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.602525 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.602533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:54.603587 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:54.603590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.604000 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.604007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:54.604079 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:54.604508 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:54.604514 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:54.606625 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
W20260924 00:14:54.606781 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:54.600838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:38167 (local address 127.1.44.190:44659)
0924 00:14:54.600945 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:54.600947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:54.600958 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:14:54.601027 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:54.601028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:54.601041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:54.601105 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:54.601595 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.602363 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.603671 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.603862 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.603925 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:54.606712 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:38167: BlockingRecv error: recv got EOF from 127.1.44.129:38167 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
Sep 24 00:14:54 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:54 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:54.733655 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:54.729307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:32939)
0924 00:14:54.729448 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:54.729461 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:54.729524 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:54.729654 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:54.729655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:54.729815 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:54.729858 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:54.729860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.730497 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.730498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:54.731033 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:54.731034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.731369 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.731370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:54.731427 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:54.731851 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:54.731855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:54.733546 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
W20260924 00:14:54.733683 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:54.729403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:32939 (local address 127.1.44.190:44659)
0924 00:14:54.729510 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:54.729512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:54.729521 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:14:54.729579 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:54.729580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:54.729590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:54.729634 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:54.729931 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.730409 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.731108 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.731311 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.731359 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:54.733622 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:32939: BlockingRecv error: recv got EOF from 127.1.44.130:32939 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 24 00:14:54 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:54 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:54.862389 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:54.857882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:55469)
0924 00:14:54.858041 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:54.858051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:54.858107 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:54.858216 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:54.858217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:54.858377 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:54.858421 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:54.858423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.859117 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.859118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:54.859705 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:54.859707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.860028 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.860029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:54.860082 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:54.860516 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:54.860520 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:54.862316 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
W20260924 00:14:54.862498 10411 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:54.857951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:55469 (local address 127.1.44.190:44659)
0924 00:14:54.858028 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:54.858031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:54.858093 (+ 62us) server_negotiation.cc:407] Connection header received
0924 00:14:54.858147 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:54.858148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:54.858159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:54.858205 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:54.858485 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.858979 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.859776 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:54.859953 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:54.860004 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:54.862409 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:55469: BlockingRecv error: recv got EOF from 127.1.44.131:55469 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 24 00:14:55 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:55 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:55.611680 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:55.607160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:35469)
0924 00:14:55.607307 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:55.607318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:55.607372 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:55.607501 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:55.607502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:55.607668 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:55.607712 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:55.607714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.608357 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.608358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:55.608911 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:55.608913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.609281 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.609282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:55.609320 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:55.609790 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:55.609795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:55.611586 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
W20260924 00:14:55.611743 10415 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:55.607262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:35469 (local address 127.1.44.190:44659)
0924 00:14:55.607357 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:55.607360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:55.607376 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:14:55.607416 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:55.607417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:55.607427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:55.607483 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:55.607779 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.608277 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.609015 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.609211 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.609273 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:55.611651 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:35469: BlockingRecv error: recv got EOF from 127.1.44.129:35469 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":47,"threads_started":1}
Sep 24 00:14:55 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:55 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:55.738950 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:55.734133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:48003)
0924 00:14:55.734331 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:55.734341 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:55.734424 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:55.734559 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:55.734560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:55.734734 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:55.734776 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:55.734777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.735549 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.735554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:55.736092 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:55.736094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.736522 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.736523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:55.736586 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:55.737072 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:55.737077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:55.738848 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":118,"threads_started":1}
W20260924 00:14:55.738982 10415 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:55.734200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48003 (local address 127.1.44.190:44659)
0924 00:14:55.734290 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:55.734292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:55.734412 (+ 120us) server_negotiation.cc:407] Connection header received
0924 00:14:55.734482 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:55.734483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:55.734496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:55.734544 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:55.734857 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.735439 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.736180 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.736432 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.736520 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:55.738920 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:48003: BlockingRecv error: recv got EOF from 127.1.44.130:48003 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 24 00:14:55 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:55 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:55.867051 10417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:55.862802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:42811)
0924 00:14:55.862945 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:55.862956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:55.863023 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:55.863133 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:55.863134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:55.863309 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:55.863353 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:55.863355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.863970 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.863971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:55.864482 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:55.864483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.864786 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.864787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:55.864826 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:55.865244 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:55.865248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:55.866995 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
W20260924 00:14:55.867189 10415 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:55.862850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:42811 (local address 127.1.44.190:44659)
0924 00:14:55.862918 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:55.862920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:55.863014 (+ 94us) server_negotiation.cc:407] Connection header received
0924 00:14:55.863066 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:55.863066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:55.863077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:55.863120 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:55.863409 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.863906 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.864549 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:55.864722 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:55.864770 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:55.867091 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:42811: BlockingRecv error: recv got EOF from 127.1.44.131:42811 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 00:14:56 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:56 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:56.616351 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:56.612082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:55131)
0924 00:14:56.612211 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:56.612220 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:56.612268 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:56.612412 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:56.612413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:56.612565 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:56.612611 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:56.612613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.613218 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.613218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:56.613733 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:56.613734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.614038 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.614039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:56.614077 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:56.614474 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:56.614478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:56.616260 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
W20260924 00:14:56.616406 10419 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:56.612155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:55131 (local address 127.1.44.190:44659)
0924 00:14:56.612289 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:56.612291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:56.612299 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:14:56.612327 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:56.612327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:56.612337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:56.612395 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:56.612675 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.613155 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.613801 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.613979 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.614026 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:56.616330 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:55131: BlockingRecv error: recv got EOF from 127.1.44.129:55131 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
Sep 24 00:14:56 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:56 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:56.743822 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:56.739466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:59237)
0924 00:14:56.739677 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:56.739690 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:56.739788 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:56.739921 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:56.739921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:56.740076 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:56.740120 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:56.740122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.740745 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.740745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:56.741251 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:56.741252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.741607 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.741608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:56.741645 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:56.742092 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:56.742096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:56.743756 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
W20260924 00:14:56.743892 10419 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:56.739559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:59237 (local address 127.1.44.190:44659)
0924 00:14:56.739626 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:56.739628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:56.739760 (+ 132us) server_negotiation.cc:407] Connection header received
0924 00:14:56.739855 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:56.739856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:56.739866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:56.739909 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:56.740178 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.740672 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.741341 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.741530 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.741600 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:56.743848 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:59237: BlockingRecv error: recv got EOF from 127.1.44.130:59237 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 00:14:56 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:56 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:56.872608 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:56.867552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:44599)
0924 00:14:56.867690 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:56.867704 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:56.867794 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:56.867965 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:56.867965 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:56.868147 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:56.868189 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:56.868191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.868852 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.868853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:56.869375 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:56.869376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.870170 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.870171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:56.870231 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:56.870751 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:56.870756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:56.872542 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
W20260924 00:14:56.872691 10419 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:56.867642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44599 (local address 127.1.44.190:44659)
0924 00:14:56.867694 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:56.867696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:56.867775 (+ 79us) server_negotiation.cc:407] Connection header received
0924 00:14:56.867867 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:56.867868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:56.867881 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:56.867956 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:56.868280 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.868770 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.869461 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:56.870089 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:56.870172 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:56.872644 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:44599: BlockingRecv error: recv got EOF from 127.1.44.131:44599 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 24 00:14:57 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:57 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:57.621402 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:57.616853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:41021)
0924 00:14:57.616989 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:57.617000 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:57.617047 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:57.617187 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:57.617188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:57.617346 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:57.617392 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:57.617394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.618079 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.618080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:57.618615 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:57.618617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.618931 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.618932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:57.618972 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:57.619427 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:57.619432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:57.621303 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
W20260924 00:14:57.621465 10423 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:57.616914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:41021 (local address 127.1.44.190:44659)
0924 00:14:57.617033 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:57.617035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:57.617045 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:57.617100 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:57.617101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:57.617111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:57.617172 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:57.617451 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.617995 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.618685 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.618872 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.618920 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:57.621394 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:41021: BlockingRecv error: recv got EOF from 127.1.44.129:41021 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
Sep 24 00:14:57 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:57 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:57.753247 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:57.744303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:54511)
0924 00:14:57.744435 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:57.744451 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:57.744505 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:57.744637 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:57.744638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:57.744796 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:57.744839 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:57.744841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.745492 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.745492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:57.746005 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:57.746006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.746309 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.746309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:57.746346 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:57.751556 (+ 5210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:57.751562 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:57.753189 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260924 00:14:57.753374 10423 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:57.744391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:54511 (local address 127.1.44.190:44659)
0924 00:14:57.744450 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:57.744452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:57.744496 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:14:57.744562 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:57.744562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:57.744573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:57.744629 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:57.744906 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.745430 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.746068 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.746252 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.746302 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:57.753293 (+ 6991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:54511: BlockingRecv error: recv got EOF from 127.1.44.130:54511 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 24 00:14:57 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:57 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.131@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:57.877507 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:57.873062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.131:52847)
0924 00:14:57.873208 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:57.873219 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:57.873280 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:57.873409 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:57.873409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:57.873574 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:57.873616 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:57.873618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.874277 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.874278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:57.874801 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:57.874802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.875145 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.875146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:57.875246 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:57.875709 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:57.875714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:57.877439 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
W20260924 00:14:57.877683 10423 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:57.873166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:52847 (local address 127.1.44.190:44659)
0924 00:14:57.873232 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:57.873238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:57.873267 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:14:57.873337 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:57.873337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:57.873350 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:57.873393 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:57.873706 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.874210 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.874875 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:57.875053 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:57.875116 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:57.877583 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.131:52847: BlockingRecv error: recv got EOF from 127.1.44.131:52847 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 24 00:14:58 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:58 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.129@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:58.626243 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:58.621894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.129:54281)
0924 00:14:58.622034 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:58.622047 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:58.622101 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:58.622251 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:58.622252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:58.622403 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:58.622450 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:58.622451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.623086 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.623087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:58.623639 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:58.623641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.623998 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.623999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:58.624049 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:58.624464 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:58.624468 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:58.626148 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
W20260924 00:14:58.626303 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:58.621968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:54281 (local address 127.1.44.190:44659)
0924 00:14:58.622088 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:58.622090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:58.622100 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:14:58.622166 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:58.622166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:58.622176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:58.622228 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:58.622508 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.623016 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.623738 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.623929 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.623993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:58.626231 (+ 2238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.129:54281: BlockingRecv error: recv got EOF from 127.1.44.129:54281 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
Sep 24 00:14:58 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 00:14:58 dist-test-slave-cm7b krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.44.130@KRBTEST.COM for oryx/127.1.44.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 00:14:58.758548 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:58.753745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44659 (local address 127.1.44.130:36807)
0924 00:14:58.753914 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:58.753927 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:58.753999 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:58.754158 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:58.754159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:58.754341 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:58.754386 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:58.754388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.755047 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.755048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:58.755650 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:58.755653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.756014 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.756015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:58.756073 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:58.756617 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:14:58.756622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:58.758433 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.44.190:44659: Server oryx/127.1.44.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
W20260924 00:14:58.758581 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:58.753848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:36807 (local address 127.1.44.190:44659)
0924 00:14:58.753922 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:58.753924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:58.753978 (+ 54us) server_negotiation.cc:407] Connection header received
0924 00:14:58.754063 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:58.754063 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:58.754076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:58.754137 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:58.754463 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.754967 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.755743 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.755944 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.756017 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:58.758517 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.44.130:36807: BlockingRecv error: recv got EOF from 127.1.44.130:36807 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 00:14:58.799301 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9952
I20260924 00:14:58.803972 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10104
I20260924 00:14:58.808156 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10237
I20260924 00:14:58.812548 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 9882
2026-09-24T00:14:58Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17414 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T00:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:58Z Disabled control of system clock
I20260924 00:14:58.839330 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35943
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:35943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 00:14:58.916213 10432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:58.916390 10432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:58.916409 10432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:58.917472 10432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:58.917507 10432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:58.917519 10432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:58.917532 10432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:58.917544 10432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:58.918772 10432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:35943
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:58.918946 10432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:58.919154 10432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:58.923629 10437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:58.923676 10440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:58.923693 10432 server_base.cc:1034] running on GCE node
W20260924 00:14:58.923663 10438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:58.924021 10432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:58.924273 10432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:58.925446 10432 hybrid_clock.cc:648] HybridClock initialized: now 1790208898925437 us; error 33 us; skew 500 ppm
I20260924 00:14:58.926563 10432 webserver.cc:460] Webserver started at http://127.1.44.190:41553/ using document root <none> and password file <none>
I20260924 00:14:58.926801 10432 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:58.926865 10432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:58.927007 10432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:58.928032 10432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "6fa87f318c384c7aa035e00431a94069"
format_stamp: "Formatted at 2026-09-24 00:14:58 on dist-test-slave-cm7b"
server_key: "7000088eaa84e03cdc46402be58b9cf2"
server_key_iv: "ff34f64f1c75442b0b5a096b31677e4d"
server_key_version: "encryptionkey@0"
I20260924 00:14:58.928375 10432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "6fa87f318c384c7aa035e00431a94069"
format_stamp: "Formatted at 2026-09-24 00:14:58 on dist-test-slave-cm7b"
server_key: "7000088eaa84e03cdc46402be58b9cf2"
server_key_iv: "ff34f64f1c75442b0b5a096b31677e4d"
server_key_version: "encryptionkey@0"
I20260924 00:14:58.929565 10432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:58.930217 10446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:58.930371 10432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:58.930467 10432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "6fa87f318c384c7aa035e00431a94069"
format_stamp: "Formatted at 2026-09-24 00:14:58 on dist-test-slave-cm7b"
server_key: "7000088eaa84e03cdc46402be58b9cf2"
server_key_iv: "ff34f64f1c75442b0b5a096b31677e4d"
server_key_version: "encryptionkey@0"
I20260924 00:14:58.930545 10432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:58.951537 10432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:58.951804 10432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:58.951943 10432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:58.955701 10432 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:35943
I20260924 00:14:58.955734 10498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:35943 every 8 connection(s)
I20260924 00:14:58.956024 10432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:58.956597 10499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:58.958940 10499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069: Bootstrap starting.
I20260924 00:14:58.959502 10499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:58.959779 10499 log.cc:824] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:58.960384 10499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069: No bootstrap required, opened a new log
I20260924 00:14:58.961823 10499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa87f318c384c7aa035e00431a94069" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35943 } }
I20260924 00:14:58.961948 10499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:58.961970 10499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fa87f318c384c7aa035e00431a94069, State: Initialized, Role: FOLLOWER
I20260924 00:14:58.962129 10499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa87f318c384c7aa035e00431a94069" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35943 } }
I20260924 00:14:58.962193 10499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:58.962241 10499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:58.962289 10499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:58.962759 10499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa87f318c384c7aa035e00431a94069" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35943 } }
I20260924 00:14:58.962842 10499 leader_election.cc:302] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fa87f318c384c7aa035e00431a94069; no voters:
I20260924 00:14:58.962985 10499 leader_election.cc:288] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:58.963078 10502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:58.963230 10499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:58.963239 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10432
I20260924 00:14:58.963259 10502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [term 1 LEADER]: Becoming Leader. State: Replica: 6fa87f318c384c7aa035e00431a94069, State: Running, Role: LEADER
I20260924 00:14:58.963318 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:58.963356 10502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa87f318c384c7aa035e00431a94069" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35943 } }
I20260924 00:14:58.963519 1202 external_mini_cluster.cc:1428] Setting key 5a2a22a480aeca16f66c6a01cfa1b6d8
I20260924 00:14:58.963747 10504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fa87f318c384c7aa035e00431a94069. Latest consensus state: current_term: 1 leader_uuid: "6fa87f318c384c7aa035e00431a94069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa87f318c384c7aa035e00431a94069" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35943 } } }
I20260924 00:14:58.963810 10503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fa87f318c384c7aa035e00431a94069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fa87f318c384c7aa035e00431a94069" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35943 } } }
I20260924 00:14:58.963927 10504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:58.963940 10503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:58.964496 10509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:58.965019 10509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:58.966782 10509 catalog_manager.cc:1347] Generated new cluster ID: 6086b9d1cc584398a98077839eb484bd
I20260924 00:14:58.966837 10509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:58.969055 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:58.964113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.1.44.190:35943)
0924 00:14:58.964327 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:58.964335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:58.964371 (+ 36us) server_negotiation.cc:407] Connection header received
0924 00:14:58.964490 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:58.964497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:58.964565 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:58.964708 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:58.964980 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.966336 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.967247 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:58.967654 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:58.967804 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:58.968469 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:58.968473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:58.968473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:58.968636 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:58.968672 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:58.968682 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:58.968754 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:58.968826 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 00:14:58.968898 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260924 00:14:58.973104 10509 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:58.973563 10509 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:58.978881 10509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fa87f318c384c7aa035e00431a94069: Generated new TSK 0
I20260924 00:14:58.979024 10509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:58.988137 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35943
--builtin_ntp_servers=127.1.44.148:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 00:14:59.070736 10523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.070919 10523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.070940 10523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.072098 10523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.072136 10523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.072160 10523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:59.073429 10523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.073642 10523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.073930 10523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.078655 10531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.078641 10528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.078759 10529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.079264 10523 server_base.cc:1034] running on GCE node
I20260924 00:14:59.079470 10523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.079712 10523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.080881 10523 hybrid_clock.cc:648] HybridClock initialized: now 1790208899080869 us; error 29 us; skew 500 ppm
I20260924 00:14:59.082084 10523 webserver.cc:460] Webserver started at http://127.1.44.129:42745/ using document root <none> and password file <none>
I20260924 00:14:59.082305 10523 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.082370 10523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.082506 10523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.083563 10523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "015591a2ff354e8bbf6d8ee67cb3db77"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "0eeeb395b2a7b6ad28763894a92a8960"
server_key_iv: "62d59f1373006406937c92c34806149f"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.083907 10523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "015591a2ff354e8bbf6d8ee67cb3db77"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "0eeeb395b2a7b6ad28763894a92a8960"
server_key_iv: "62d59f1373006406937c92c34806149f"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.085250 10523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:59.085959 10537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.086177 10523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:14:59.086277 10523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "015591a2ff354e8bbf6d8ee67cb3db77"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "0eeeb395b2a7b6ad28763894a92a8960"
server_key_iv: "62d59f1373006406937c92c34806149f"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.086340 10523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.097892 10523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.098162 10523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.098304 10523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.098527 10523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:59.098855 10523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:59.098901 10523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.098927 10523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:59.098943 10523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.104518 10523 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:37405
I20260924 00:14:59.104583 10650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:37405 every 8 connection(s)
I20260924 00:14:59.104877 10523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:59.109320 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.105528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:43641 (local address 127.1.44.190:35943)
0924 00:14:59.105595 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.105598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.105807 (+ 209us) server_negotiation.cc:407] Connection header received
0924 00:14:59.106227 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.106228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.106238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.106288 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.106848 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.107377 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.108258 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.108430 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.108479 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:59.109114 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.109115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.109115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.109119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.109130 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.109134 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.109185 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.109243 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.109267 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:14:59.109516 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.105508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35943 (local address 127.1.44.129:43641)
0924 00:14:59.105697 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:59.105725 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:59.106153 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:59.106335 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:59.106338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:59.106345 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:59.106785 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:59.106788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.107454 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.107455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:59.108192 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:59.108193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.108495 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.108496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:59.108592 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:59.109002 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:59.109002 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:59.109026 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:59.109045 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:59.109188 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:59.109190 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:59.109192 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:59.109235 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 00:14:59.109303 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260924 00:14:59.110080 10651 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35943
I20260924 00:14:59.110162 10651 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:59.110322 10651 heartbeater.cc:511] Master 127.1.44.190:35943 requested a full tablet report, sending...
I20260924 00:14:59.110778 10463 ts_manager.cc:194] Registered new tserver with Master: 015591a2ff354e8bbf6d8ee67cb3db77 (127.1.44.129:37405)
I20260924 00:14:59.111410 10463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.129:43641
I20260924 00:14:59.112870 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10523
I20260924 00:14:59.112941 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:59.113149 1202 external_mini_cluster.cc:1428] Setting key 24c499bf988d9c87025c12be8300a34a
I20260924 00:14:59.113945 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35943
--builtin_ntp_servers=127.1.44.148:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 00:14:59.191363 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.191540 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.191560 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.192658 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.192696 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.192720 10654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:59.193995 10654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.194216 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.194483 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.199079 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.199095 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.199083 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.199112 10654 server_base.cc:1034] running on GCE node
I20260924 00:14:59.200335 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.200567 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.201713 10654 hybrid_clock.cc:648] HybridClock initialized: now 1790208899201702 us; error 39 us; skew 500 ppm
I20260924 00:14:59.202914 10654 webserver.cc:460] Webserver started at http://127.1.44.130:40009/ using document root <none> and password file <none>
I20260924 00:14:59.203148 10654 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.203230 10654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.203361 10654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.204320 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "215761d735d340f6b540ec9bbab85bb8"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "8ea52bc64e7808482ea3ff0083eced2b"
server_key_iv: "f47bfcae4e4842d9faec177b8dd59595"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.204655 10654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "215761d735d340f6b540ec9bbab85bb8"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "8ea52bc64e7808482ea3ff0083eced2b"
server_key_iv: "f47bfcae4e4842d9faec177b8dd59595"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.205854 10654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:59.206485 10668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.206642 10654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:59.206732 10654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "215761d735d340f6b540ec9bbab85bb8"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "8ea52bc64e7808482ea3ff0083eced2b"
server_key_iv: "f47bfcae4e4842d9faec177b8dd59595"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.206813 10654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.218866 10654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.219106 10654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.219288 10654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.219477 10654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:59.219753 10654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:59.219810 10654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.219848 10654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:59.219875 10654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.225245 10654 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:43573
I20260924 00:14:59.225309 10781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:43573 every 8 connection(s)
I20260924 00:14:59.225589 10654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:59.228076 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10654
I20260924 00:14:59.228173 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:59.228350 1202 external_mini_cluster.cc:1428] Setting key a48f01ec645222620489d52aa9c6c701
I20260924 00:14:59.229199 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35943
--builtin_ntp_servers=127.1.44.148:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 00:14:59.230595 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.226297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:48365 (local address 127.1.44.190:35943)
0924 00:14:59.226377 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.226379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.226533 (+ 154us) server_negotiation.cc:407] Connection header received
0924 00:14:59.226993 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.226994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.227006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.227057 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.227708 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.228216 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.229312 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.229520 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.229591 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:59.230375 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.230376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.230376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.230380 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.230394 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.230398 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.230446 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.230521 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.230543 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 00:14:59.230844 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.226196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35943 (local address 127.1.44.130:48365)
0924 00:14:59.226413 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:59.226449 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:59.226893 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:59.227115 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:59.227119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:59.227128 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:59.227597 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:59.227601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.228301 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.228302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:59.229231 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:59.229233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.229596 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.229597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:59.229725 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:59.230225 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:59.230226 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:59.230264 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:59.230295 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:59.230449 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:59.230452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:59.230455 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:59.230525 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 00:14:59.230623 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260924 00:14:59.231292 10782 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35943
I20260924 00:14:59.231369 10782 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:59.231519 10782 heartbeater.cc:511] Master 127.1.44.190:35943 requested a full tablet report, sending...
I20260924 00:14:59.231870 10463 ts_manager.cc:194] Registered new tserver with Master: 215761d735d340f6b540ec9bbab85bb8 (127.1.44.130:43573)
I20260924 00:14:59.232272 10463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.130:48365
W20260924 00:14:59.307662 10785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.307832 10785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.307860 10785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.308890 10785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.308919 10785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.308943 10785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:59.310251 10785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.310412 10785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.310622 10785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.315218 10793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.315260 10785 server_base.cc:1034] running on GCE node
W20260924 00:14:59.315248 10790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.315232 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.315658 10785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.315879 10785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.317015 10785 hybrid_clock.cc:648] HybridClock initialized: now 1790208899317003 us; error 31 us; skew 500 ppm
I20260924 00:14:59.318218 10785 webserver.cc:460] Webserver started at http://127.1.44.131:38501/ using document root <none> and password file <none>
I20260924 00:14:59.318450 10785 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.318495 10785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.318639 10785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.319657 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "fd757636a5e44cf38006d2e24e10890f"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "c300cb7a5f351c7563284e584f9eec81"
server_key_iv: "0950da7e643c00b6193af5f305f146c1"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.319988 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "fd757636a5e44cf38006d2e24e10890f"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "c300cb7a5f351c7563284e584f9eec81"
server_key_iv: "0950da7e643c00b6193af5f305f146c1"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.321151 10785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:59.321772 10799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.322038 10785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:59.322121 10785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "fd757636a5e44cf38006d2e24e10890f"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "c300cb7a5f351c7563284e584f9eec81"
server_key_iv: "0950da7e643c00b6193af5f305f146c1"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.322189 10785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.336714 10785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.336977 10785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.337112 10785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.337313 10785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:59.337599 10785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:59.337649 10785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.337688 10785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:59.337715 10785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.343261 10912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:44603 every 8 connection(s)
I20260924 00:14:59.343261 10785 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:44603
I20260924 00:14:59.343660 10785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:59.348565 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.344336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:37713 (local address 127.1.44.190:35943)
0924 00:14:59.344390 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.344393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.344608 (+ 215us) server_negotiation.cc:407] Connection header received
0924 00:14:59.345115 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.345116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.345128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.345172 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.345783 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.346281 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.347255 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.347458 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.347539 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:59.348303 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.348304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.348305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.348310 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.348324 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.348328 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.348376 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.348460 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.348490 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:14:59.348780 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.344330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35943 (local address 127.1.44.131:37713)
0924 00:14:59.344505 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:59.344534 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:59.345016 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:59.345231 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:59.345234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:59.345247 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:59.345691 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:59.345695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.346368 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.346370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:59.347116 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:14:59.347118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:14:59.347547 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:14:59.347548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:14:59.347668 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:14:59.348170 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:59.348171 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:59.348196 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:59.348216 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:59.348374 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:59.348379 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:59.348382 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:59.348462 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 00:14:59.348553 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260924 00:14:59.349220 10913 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35943
I20260924 00:14:59.349293 10913 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:59.349414 10913 heartbeater.cc:511] Master 127.1.44.190:35943 requested a full tablet report, sending...
I20260924 00:14:59.349711 10463 ts_manager.cc:194] Registered new tserver with Master: fd757636a5e44cf38006d2e24e10890f (127.1.44.131:44603)
I20260924 00:14:59.350070 10463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.131:37713
I20260924 00:14:59.353865 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10785
I20260924 00:14:59.354068 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:59.354261 1202 external_mini_cluster.cc:1428] Setting key e92ae150751f365f4902647265b4c6ab
I20260924 00:14:59.355038 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 00:14:59.356742 10511 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:59.356313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.1.44.190:35943)
0924 00:14:59.356347 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.356349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.356475 (+ 126us) server_negotiation.cc:407] Connection header received
0924 00:14:59.356525 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.356526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.356536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.356574 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.356681 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48522: BlockingRecv error: recv got EOF from 127.0.0.1:48522 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:14:59.356967 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10523
I20260924 00:14:59.361970 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10654
I20260924 00:14:59.366061 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10785
I20260924 00:14:59.369949 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10432
2026-09-24T00:14:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (555 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T00:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:14:59Z Disabled control of system clock
I20260924 00:14:59.395092 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:33249
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:40825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:33249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 00:14:59.471858 10924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.472038 10924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.472059 10924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.473078 10924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:14:59.473110 10924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.473124 10924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.473135 10924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:14:59.473147 10924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:14:59.474349 10924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:33249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:33249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.474525 10924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.474741 10924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.479449 10932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.479458 10930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.479485 10924 server_base.cc:1034] running on GCE node
W20260924 00:14:59.479460 10929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.479866 10924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.480127 10924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.481283 10924 hybrid_clock.cc:648] HybridClock initialized: now 1790208899481263 us; error 32 us; skew 500 ppm
I20260924 00:14:59.482348 10924 webserver.cc:460] Webserver started at http://127.1.44.190:37849/ using document root <none> and password file <none>
I20260924 00:14:59.482569 10924 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.482632 10924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.482771 10924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.483767 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "fda7b403ab264d4e8dfe989b7278a597"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "59db4a6f7dddda61b506b15cdf9c00de"
server_key_iv: "fa618708dd6b69c5b4c96fdc0b5e037b"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.484097 10924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "fda7b403ab264d4e8dfe989b7278a597"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "59db4a6f7dddda61b506b15cdf9c00de"
server_key_iv: "fa618708dd6b69c5b4c96fdc0b5e037b"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.485327 10924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:59.485973 10938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.486133 10924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:59.486229 10924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "fda7b403ab264d4e8dfe989b7278a597"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "59db4a6f7dddda61b506b15cdf9c00de"
server_key_iv: "fa618708dd6b69c5b4c96fdc0b5e037b"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.486304 10924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.489465 10924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.489712 10924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.489851 10924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.493393 10924 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:33249
I20260924 00:14:59.493425 10990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:33249 every 8 connection(s)
I20260924 00:14:59.493713 10924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:14:59.494269 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:14:59.496701 10991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597: Bootstrap starting.
I20260924 00:14:59.497184 10991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597: Neither blocks nor log segments found. Creating new log.
I20260924 00:14:59.497418 10991 log.cc:824] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597: Log is configured to *not* fsync() on all Append() calls
I20260924 00:14:59.497972 10991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597: No bootstrap required, opened a new log
I20260924 00:14:59.498634 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 10924
I20260924 00:14:59.498755 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:14:59.498936 1202 external_mini_cluster.cc:1428] Setting key 73f1604557f7f04b9f2c9b76f5b62af4
I20260924 00:14:59.499892 10991 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda7b403ab264d4e8dfe989b7278a597" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33249 } }
I20260924 00:14:59.500052 10991 raft_consensus.cc:374] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:14:59.500088 10991 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda7b403ab264d4e8dfe989b7278a597, State: Initialized, Role: FOLLOWER
I20260924 00:14:59.500533 10991 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda7b403ab264d4e8dfe989b7278a597" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33249 } }
I20260924 00:14:59.500653 10991 raft_consensus.cc:388] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:14:59.500689 10991 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:14:59.500736 10991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:14:59.501137 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.499434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.1.44.190:33249)
0924 00:14:59.499641 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.499650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.499684 (+ 34us) server_negotiation.cc:407] Connection header received
0924 00:14:59.499862 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.499869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.499921 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.500054 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.500672 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.500676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.500676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.500680 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.500723 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.500737 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.500808 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.500889 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.500942 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260924 00:14:59.501739 10991 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda7b403ab264d4e8dfe989b7278a597" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33249 } }
I20260924 00:14:59.501856 10991 leader_election.cc:302] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fda7b403ab264d4e8dfe989b7278a597; no voters:
I20260924 00:14:59.502018 10991 leader_election.cc:288] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:14:59.502060 10996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:14:59.502187 10996 raft_consensus.cc:686] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [term 1 LEADER]: Becoming Leader. State: Replica: fda7b403ab264d4e8dfe989b7278a597, State: Running, Role: LEADER
I20260924 00:14:59.502236 10991 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:14:59.502276 10996 consensus_queue.cc:237] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda7b403ab264d4e8dfe989b7278a597" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33249 } }
I20260924 00:14:59.502580 10996 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda7b403ab264d4e8dfe989b7278a597. Latest consensus state: current_term: 1 leader_uuid: "fda7b403ab264d4e8dfe989b7278a597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda7b403ab264d4e8dfe989b7278a597" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33249 } } }
I20260924 00:14:59.502590 10998 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fda7b403ab264d4e8dfe989b7278a597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda7b403ab264d4e8dfe989b7278a597" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33249 } } }
I20260924 00:14:59.502693 10996 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:59.502782 10998 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597 [sys.catalog]: This master's current role is: LEADER
I20260924 00:14:59.502997 11005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:14:59.503516 11005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:14:59.505074 11005 catalog_manager.cc:1347] Generated new cluster ID: 5d159f98885d414d8dd4ecf3732c1efd
I20260924 00:14:59.505125 11005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:14:59.516197 11005 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:14:59.516340 11005 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:14:59.530787 11005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fda7b403ab264d4e8dfe989b7278a597: Generated new TSK 0
I20260924 00:14:59.530947 11005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:14:59.540417 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33249
--builtin_ntp_servers=127.1.44.148:40825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 00:14:59.636380 11015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.636591 11015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.636626 11015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.637722 11015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.637782 11015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.637825 11015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:14:59.639099 11015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.639353 11015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.639623 11015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.645068 11020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.645071 11023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.645152 11021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.645255 11015 server_base.cc:1034] running on GCE node
I20260924 00:14:59.645475 11015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.645728 11015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.646888 11015 hybrid_clock.cc:648] HybridClock initialized: now 1790208899646874 us; error 37 us; skew 500 ppm
I20260924 00:14:59.648054 11015 webserver.cc:460] Webserver started at http://127.1.44.129:42169/ using document root <none> and password file <none>
I20260924 00:14:59.648278 11015 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.648341 11015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.648468 11015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.649444 11015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "558b4fdf999b4e9e8fc1cdd79f1f4a0c"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "268e9a593b3aad6dad03afed26e577e0"
server_key_iv: "0f5f71c0e2947fbb1a301d88fc3e5214"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.649793 11015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "558b4fdf999b4e9e8fc1cdd79f1f4a0c"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "268e9a593b3aad6dad03afed26e577e0"
server_key_iv: "0f5f71c0e2947fbb1a301d88fc3e5214"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.651081 11015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:14:59.651834 11029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.652012 11015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.652102 11015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "558b4fdf999b4e9e8fc1cdd79f1f4a0c"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "268e9a593b3aad6dad03afed26e577e0"
server_key_iv: "0f5f71c0e2947fbb1a301d88fc3e5214"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.652181 11015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.655297 11015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.655527 11015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.655656 11015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.655856 11015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:59.656129 11015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:59.656176 11015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.656214 11015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:59.656250 11015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.661635 11015 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:41537
I20260924 00:14:59.661666 11142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:41537 every 8 connection(s)
I20260924 00:14:59.661954 11015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:14:59.664680 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.662567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:53419 (local address 127.1.44.190:33249)
0924 00:14:59.662620 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.662622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.662843 (+ 221us) server_negotiation.cc:407] Connection header received
0924 00:14:59.663299 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.663301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.663307 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.663362 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.664456 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.664457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.664457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.664462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.664477 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.664481 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.664533 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.664602 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.664616 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 00:14:59.664883 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.662569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33249 (local address 127.1.44.129:53419)
0924 00:14:59.662741 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:59.662768 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:59.663175 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:59.663427 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:59.663430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:59.663439 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:59.664299 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:59.664300 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:59.664325 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:59.664350 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:59.664530 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:59.664535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:59.664538 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:59.664608 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 00:14:59.664663 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260924 00:14:59.665380 11143 heartbeater.cc:347] Connected to a master server at 127.1.44.190:33249
I20260924 00:14:59.665474 11143 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:59.665520 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11015
I20260924 00:14:59.665602 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:14:59.665622 11143 heartbeater.cc:511] Master 127.1.44.190:33249 requested a full tablet report, sending...
I20260924 00:14:59.665792 1202 external_mini_cluster.cc:1428] Setting key 0ca4b07311108747872985c70ccf5dca
I20260924 00:14:59.666117 10955 ts_manager.cc:194] Registered new tserver with Master: 558b4fdf999b4e9e8fc1cdd79f1f4a0c (127.1.44.129:41537)
I20260924 00:14:59.666823 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33249
--builtin_ntp_servers=127.1.44.148:40825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 00:14:59.747426 11146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.747604 11146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.747625 11146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.748677 11146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.748708 11146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.748739 11146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:14:59.749963 11146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.750128 11146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.750339 11146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.755002 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.755098 11154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.755028 11146 server_base.cc:1034] running on GCE node
W20260924 00:14:59.755085 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.755441 11146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.755674 11146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.756825 11146 hybrid_clock.cc:648] HybridClock initialized: now 1790208899756817 us; error 33 us; skew 500 ppm
I20260924 00:14:59.758096 11146 webserver.cc:460] Webserver started at http://127.1.44.130:39653/ using document root <none> and password file <none>
I20260924 00:14:59.758324 11146 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.758369 11146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.758522 11146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.759567 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "2e9a2c7f741a4783a1f27d02f86a86ab"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "793793918798362e95281148fa0d7654"
server_key_iv: "c9bbe4ddc6acfbe8c812c4d8fa08c992"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.759902 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "2e9a2c7f741a4783a1f27d02f86a86ab"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "793793918798362e95281148fa0d7654"
server_key_iv: "c9bbe4ddc6acfbe8c812c4d8fa08c992"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.761085 11146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:14:59.761785 11160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.761974 11146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:14:59.762056 11146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "2e9a2c7f741a4783a1f27d02f86a86ab"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "793793918798362e95281148fa0d7654"
server_key_iv: "c9bbe4ddc6acfbe8c812c4d8fa08c992"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.762122 11146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.765285 11146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.765511 11146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.765637 11146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.765831 11146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:59.766112 11146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:59.766149 11146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.766198 11146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:59.766227 11146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.771579 11146 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:32819
I20260924 00:14:59.771641 11273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:32819 every 8 connection(s)
I20260924 00:14:59.771910 11146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:14:59.774559 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.772560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:35349 (local address 127.1.44.190:33249)
0924 00:14:59.772613 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.772615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.772805 (+ 190us) server_negotiation.cc:407] Connection header received
0924 00:14:59.773240 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.773241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.773247 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.773290 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.774309 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.774311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.774312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.774316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.774333 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.774337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.774399 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.774478 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.774493 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 00:14:59.774735 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.772556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33249 (local address 127.1.44.130:35349)
0924 00:14:59.772702 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:59.772730 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:59.773147 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:59.773363 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:59.773366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:59.773373 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:59.774169 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:59.774169 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:59.774194 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:59.774217 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:59.774397 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:59.774401 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:59.774404 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:14:59.774486 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 00:14:59.774540 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260924 00:14:59.775138 11274 heartbeater.cc:347] Connected to a master server at 127.1.44.190:33249
I20260924 00:14:59.775246 11274 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:59.775391 11274 heartbeater.cc:511] Master 127.1.44.190:33249 requested a full tablet report, sending...
I20260924 00:14:59.775692 10954 ts_manager.cc:194] Registered new tserver with Master: 2e9a2c7f741a4783a1f27d02f86a86ab (127.1.44.130:32819)
I20260924 00:14:59.781000 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11146
I20260924 00:14:59.781118 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:14:59.781318 1202 external_mini_cluster.cc:1428] Setting key 531db9bbadb21c04bf023b62d0275c7e
I20260924 00:14:59.782147 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33249
--builtin_ntp_servers=127.1.44.148:40825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 00:14:59.860128 11277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:14:59.860335 11277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:14:59.860369 11277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:14:59.861490 11277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:14:59.861548 11277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:14:59.861588 11277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:14:59.862876 11277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:14:59.863122 11277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:14:59.863405 11277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:14:59.868113 11282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:14:59.868218 11285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.868183 11277 server_base.cc:1034] running on GCE node
W20260924 00:14:59.868367 11283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:14:59.868589 11277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:14:59.868783 11277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:14:59.869913 11277 hybrid_clock.cc:648] HybridClock initialized: now 1790208899869901 us; error 27 us; skew 500 ppm
I20260924 00:14:59.870975 11277 webserver.cc:460] Webserver started at http://127.1.44.131:35595/ using document root <none> and password file <none>
I20260924 00:14:59.871143 11277 fs_manager.cc:359] Metadata directory not provided
I20260924 00:14:59.871181 11277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:14:59.871328 11277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:14:59.872251 11277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "5ee677914fcc4a4c8f05419eebdce0bd"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "7deae4319e1bb96ac9aaf64de7176451"
server_key_iv: "0f7338650c9f30bcd37c41447101681a"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.872532 11277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "5ee677914fcc4a4c8f05419eebdce0bd"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "7deae4319e1bb96ac9aaf64de7176451"
server_key_iv: "0f7338650c9f30bcd37c41447101681a"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.873670 11277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:14:59.874284 11291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.874439 11277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:14:59.874513 11277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "5ee677914fcc4a4c8f05419eebdce0bd"
format_stamp: "Formatted at 2026-09-24 00:14:59 on dist-test-slave-cm7b"
server_key: "7deae4319e1bb96ac9aaf64de7176451"
server_key_iv: "0f7338650c9f30bcd37c41447101681a"
server_key_version: "encryptionkey@0"
I20260924 00:14:59.874562 11277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:14:59.877660 11277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:14:59.877846 11277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:14:59.877933 11277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:14:59.878105 11277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:14:59.878388 11277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:14:59.878420 11277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.878439 11277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:14:59.878451 11277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:14:59.883569 11277 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:43885
I20260924 00:14:59.883627 11404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:43885 every 8 connection(s)
I20260924 00:14:59.883898 11277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:14:59.886088 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11277
I20260924 00:14:59.886160 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:14:59.886343 1202 external_mini_cluster.cc:1428] Setting key 57c0ce1bb4319340e380dc67cd3d4e7b
I20260924 00:14:59.886521 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.884524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:36981 (local address 127.1.44.190:33249)
0924 00:14:59.884579 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.884581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.884774 (+ 193us) server_negotiation.cc:407] Connection header received
0924 00:14:59.885140 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.885140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.885147 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.885198 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.886248 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:14:59.886250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:14:59.886250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:14:59.886254 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:14:59.886267 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:14:59.886271 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:14:59.886317 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 00:14:59.886432 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 00:14:59.886451 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 00:14:59.886675 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:14:59.884505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33249 (local address 127.1.44.131:36981)
0924 00:14:59.884666 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 00:14:59.884699 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:14:59.885066 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:14:59.885240 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:14:59.885242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:14:59.885250 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:14:59.886077 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:14:59.886077 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:14:59.886100 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:14:59.886123 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:14:59.886325 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:14:59.886330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:14:59.886332 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:14:59.886421 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 00:14:59.886477 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260924 00:14:59.887104 11405 heartbeater.cc:347] Connected to a master server at 127.1.44.190:33249
I20260924 00:14:59.887187 11405 heartbeater.cc:464] Registering TS with master...
I20260924 00:14:59.887365 11405 heartbeater.cc:511] Master 127.1.44.190:33249 requested a full tablet report, sending...
I20260924 00:14:59.887677 10954 ts_manager.cc:194] Registered new tserver with Master: 5ee677914fcc4a4c8f05419eebdce0bd (127.1.44.131:43885)
I20260924 00:14:59.897843 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 00:14:59.899551 10995 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:14:59.899020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.1.44.190:33249)
0924 00:14:59.899067 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:14:59.899069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:14:59.899215 (+ 146us) server_negotiation.cc:407] Connection header received
0924 00:14:59.899302 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:14:59.899302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:14:59.899307 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:14:59.899368 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:14:59.899491 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40036: BlockingRecv error: recv got EOF from 127.0.0.1:40036 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 00:14:59.899605 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11015
I20260924 00:14:59.904500 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11146
I20260924 00:14:59.908932 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11277
I20260924 00:14:59.912675 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 10924
2026-09-24T00:14:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (542 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:14:59 dist-test-slave-cm7b krb5kdc[11415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:14:59 dist-test-slave-cm7b krb5kdc[11415](info): set up 2 sockets
Sep 24 00:14:59 dist-test-slave-cm7b krb5kdc[11415](info): commencing operation
krb5kdc: starting...
W20260924 00:15:00.956068 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:15:01.065874 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44457
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:01.144207 11428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:01.144443 11428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:01.144479 11428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:01.145618 11428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:15:01.145681 11428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:01.145709 11428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:01.145740 11428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:15:01.145761 11428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:15:01.147249 11428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:01.147513 11428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:01.147732 11428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:01.152480 11434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:01.152500 11433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:01.152499 11436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:01.152580 11428 server_base.cc:1034] running on GCE node
I20260924 00:15:01.152952 11428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:01.153221 11428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:01.154402 11428 hybrid_clock.cc:648] HybridClock initialized: now 1790208901154386 us; error 26 us; skew 500 ppm
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:01.156803 11428 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:15:01.157158 11428 webserver.cc:460] Webserver started at http://127.1.44.190:33327/ using document root <none> and password file <none>
I20260924 00:15:01.157338 11428 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:01.157377 11428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:01.157460 11428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:15:01.158398 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "e2b55bf3bde547229819ec2ac0e20425"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "8ca645c1d6661dde7efe3328e26a7815"
server_key_iv: "f01564ea34353951f43b0368c37283d7"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.158684 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "e2b55bf3bde547229819ec2ac0e20425"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "8ca645c1d6661dde7efe3328e26a7815"
server_key_iv: "f01564ea34353951f43b0368c37283d7"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.159948 11428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:15:01.160595 11443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.160758 11428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:15:01.160836 11428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "e2b55bf3bde547229819ec2ac0e20425"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "8ca645c1d6661dde7efe3328e26a7815"
server_key_iv: "f01564ea34353951f43b0368c37283d7"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.160894 11428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:01.167305 11428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:01.168931 11428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:01.169027 11428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:01.172583 11428 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44457
I20260924 00:15:01.172626 11495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44457 every 8 connection(s)
I20260924 00:15:01.172917 11428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:15:01.173821 11496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:01.175978 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425: Bootstrap starting.
I20260924 00:15:01.176517 11496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:01.176780 11496 log.cc:824] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:01.177402 11496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425: No bootstrap required, opened a new log
I20260924 00:15:01.178887 11496 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b55bf3bde547229819ec2ac0e20425" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44457 } }
I20260924 00:15:01.179035 11496 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:01.179080 11496 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b55bf3bde547229819ec2ac0e20425, State: Initialized, Role: FOLLOWER
I20260924 00:15:01.179248 11496 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b55bf3bde547229819ec2ac0e20425" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44457 } }
I20260924 00:15:01.179343 11496 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:15:01.179392 11496 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:15:01.179441 11496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:01.179684 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11428
I20260924 00:15:01.179781 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:15:01.180004 1202 external_mini_cluster.cc:1428] Setting key a68c6febfc4c37f454d41902c840523f
I20260924 00:15:01.180490 11496 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b55bf3bde547229819ec2ac0e20425" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44457 } }
I20260924 00:15:01.180598 11496 leader_election.cc:302] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2b55bf3bde547229819ec2ac0e20425; no voters:
I20260924 00:15:01.180753 11496 leader_election.cc:288] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:15:01.180774 11501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:15:01.180877 11501 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [term 1 LEADER]: Becoming Leader. State: Replica: e2b55bf3bde547229819ec2ac0e20425, State: Running, Role: LEADER
I20260924 00:15:01.180963 11501 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b55bf3bde547229819ec2ac0e20425" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44457 } }
I20260924 00:15:01.180992 11496 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:15:01.181291 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2b55bf3bde547229819ec2ac0e20425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b55bf3bde547229819ec2ac0e20425" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44457 } } }
I20260924 00:15:01.181350 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [sys.catalog]: This master's current role is: LEADER
I20260924 00:15:01.181463 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2b55bf3bde547229819ec2ac0e20425. Latest consensus state: current_term: 1 leader_uuid: "e2b55bf3bde547229819ec2ac0e20425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2b55bf3bde547229819ec2ac0e20425" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44457 } } }
I20260924 00:15:01.181519 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425 [sys.catalog]: This master's current role is: LEADER
I20260924 00:15:01.181629 11505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:15:01.182415 11505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:01.187636 11505 catalog_manager.cc:1347] Generated new cluster ID: 4582320ece0f472491ea9e9c48f1840d
I20260924 00:15:01.187690 11505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:15:01.190260 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.180476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.1.44.190:44457)
0924 00:15:01.180685 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.180694 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.180724 (+ 30us) server_negotiation.cc:407] Connection header received
0924 00:15:01.180853 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.180860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.180921 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.181062 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:01.181873 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.183046 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.184016 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.184425 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.184548 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.186909 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:01.186913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:01.186914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:01.186928 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:01.188731 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.189068 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.189070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.189071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.189173 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.189370 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.189371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.189371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.189591 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:01.189756 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.189846 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.190009 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260924 00:15:01.216634 11505 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:15:01.217165 11505 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:15:01.221361 11505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425: Generated new TSK 0
I20260924 00:15:01.221525 11505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:15:01.249926 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44457
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:01.344817 11522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:01.345005 11522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:01.345027 11522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:01.346086 11522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:01.346120 11522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:01.346145 11522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:15:01.347560 11522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:01.347730 11522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:01.347890 11522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:01.352488 11527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:01.352488 11530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:01.352514 11528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:01.352607 11522 server_base.cc:1034] running on GCE node
I20260924 00:15:01.352926 11522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:01.353156 11522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:01.354285 11522 hybrid_clock.cc:648] HybridClock initialized: now 1790208901354273 us; error 26 us; skew 500 ppm
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:01.356593 11522 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:15:01.356984 11522 webserver.cc:460] Webserver started at http://127.1.44.129:45783/ using document root <none> and password file <none>
I20260924 00:15:01.357193 11522 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:01.357237 11522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:01.357369 11522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:15:01.358404 11522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "be7c82cd444447ebbb47aa2bbe753a42"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "f6000dc473bfb3769d137c8ec23dade0"
server_key_iv: "386f59fbeb4b5c1c384140eb421336a6"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.358728 11522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "be7c82cd444447ebbb47aa2bbe753a42"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "f6000dc473bfb3769d137c8ec23dade0"
server_key_iv: "386f59fbeb4b5c1c384140eb421336a6"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.360015 11522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:15:01.360649 11537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.360798 11522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:15:01.360908 11522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "be7c82cd444447ebbb47aa2bbe753a42"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "f6000dc473bfb3769d137c8ec23dade0"
server_key_iv: "386f59fbeb4b5c1c384140eb421336a6"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.360976 11522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:01.375105 11522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:01.376748 11522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:01.376895 11522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:01.377099 11522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:15:01.377363 11522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:15:01.377413 11522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.377451 11522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:15:01.377478 11522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.383436 11522 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:37591
I20260924 00:15:01.383486 11650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:37591 every 8 connection(s)
I20260924 00:15:01.383755 11522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:15:01.384567 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11522
I20260924 00:15:01.384665 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:15:01.384856 1202 external_mini_cluster.cc:1428] Setting key dc2a27ee5995995cb73956a4e81787ca
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:15:01.393410 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.384415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:38005 (local address 127.1.44.190:44457)
0924 00:15:01.384482 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.384484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.384694 (+ 210us) server_negotiation.cc:407] Connection header received
0924 00:15:01.385328 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.385329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.385341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.385379 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:01.386602 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.387082 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.388088 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.388294 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.388364 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.390686 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:01.390688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:01.390688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:01.390717 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:01.392091 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.392479 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.392480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.392481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.392537 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.392778 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.392779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.392779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.392906 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:01.392972 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.393239 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.393321 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:15:01.393567 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.384374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44457 (local address 127.1.44.129:38005)
0924 00:15:01.384563 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.384608 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.385232 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.385440 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.385453 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.386249 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:15:01.386513 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.386519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.387178 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.387180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.387995 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.387997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.388383 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.388384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.388504 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.389035 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:15:01.389043 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:15:01.390591 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:15:01.392182 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:01.392189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:01.392191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:01.392405 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:01.392602 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:01.392603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:01.392603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:01.392707 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:01.392982 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:15:01.392988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:15:01.393112 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.393205 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.393350 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260924 00:15:01.394001 11651 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44457
I20260924 00:15:01.394078 11651 heartbeater.cc:464] Registering TS with master...
I20260924 00:15:01.394253 11651 heartbeater.cc:511] Master 127.1.44.190:44457 requested a full tablet report, sending...
I20260924 00:15:01.394724 11460 ts_manager.cc:194] Registered new tserver with Master: be7c82cd444447ebbb47aa2bbe753a42 (127.1.44.129:37591)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
I20260924 00:15:01.395570 11460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:38005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:15:01.409418 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44457
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:01.486361 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:01.486567 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:01.486606 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:01.487776 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:01.487833 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:01.487879 11656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:15:01.489352 11656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:01.489596 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:01.489815 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:01.494438 11664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:01.494443 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:01.494470 11656 server_base.cc:1034] running on GCE node
W20260924 00:15:01.494469 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:01.494835 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:01.495054 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:01.496201 11656 hybrid_clock.cc:648] HybridClock initialized: now 1790208901496195 us; error 35 us; skew 500 ppm
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:01.498534 11656 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:15:01.498926 11656 webserver.cc:460] Webserver started at http://127.1.44.130:44961/ using document root <none> and password file <none>
I20260924 00:15:01.499115 11656 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:01.499154 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:01.499284 11656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:15:01.500173 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "e843a309669744b8bef439e9faae5aaf"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "563865e4e4811d331d4fd0f47edc085f"
server_key_iv: "91470e46f9d40692b71610c9979336df"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.500447 11656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "e843a309669744b8bef439e9faae5aaf"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "563865e4e4811d331d4fd0f47edc085f"
server_key_iv: "91470e46f9d40692b71610c9979336df"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.501596 11656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:15:01.502187 11671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.502331 11656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:15:01.502417 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "e843a309669744b8bef439e9faae5aaf"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "563865e4e4811d331d4fd0f47edc085f"
server_key_iv: "91470e46f9d40692b71610c9979336df"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.502467 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:01.517827 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:01.519513 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:01.519618 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:01.519796 11656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:15:01.520105 11656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:15:01.520139 11656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.520179 11656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:15:01.520205 11656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.525563 11656 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:43767
I20260924 00:15:01.525624 11784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:43767 every 8 connection(s)
I20260924 00:15:01.525914 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:01.533524 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11656
I20260924 00:15:01.533605 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:15:01.533808 1202 external_mini_cluster.cc:1428] Setting key 7c124fceceab37193765fade54f62275
I20260924 00:15:01.534660 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.526579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:39139 (local address 127.1.44.190:44457)
0924 00:15:01.526651 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.526654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.526810 (+ 156us) server_negotiation.cc:407] Connection header received
0924 00:15:01.527322 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.527323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.527335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.527397 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:01.528267 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.528764 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.529717 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.529930 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.530018 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.531973 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:01.531974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:01.531974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:01.531980 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:01.533277 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.533726 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.533728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.533728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.533785 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.534059 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.534060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.534060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.534170 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:01.534238 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.534495 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.534574 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 00:15:01.534840 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.526534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44457 (local address 127.1.44.130:39139)
0924 00:15:01.526706 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.526737 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.527234 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.527461 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.527470 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.528004 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:15:01.528174 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.528178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.528859 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.528859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.529632 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.529634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.530025 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.530026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.530142 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.530569 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:15:01.530576 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:15:01.531868 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:15:01.533394 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:01.533400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:01.533401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:01.533661 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:01.533859 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:01.533860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:01.533860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:01.533995 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:01.534240 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:15:01.534246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:15:01.534369 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.534467 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.534611 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260924 00:15:01.535295 11785 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44457
I20260924 00:15:01.535393 11785 heartbeater.cc:464] Registering TS with master...
I20260924 00:15:01.535583 11785 heartbeater.cc:511] Master 127.1.44.190:44457 requested a full tablet report, sending...
I20260924 00:15:01.535993 11460 ts_manager.cc:194] Registered new tserver with Master: e843a309669744b8bef439e9faae5aaf (127.1.44.130:43767)
I20260924 00:15:01.536355 11460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:39139
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:15:01.558454 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44457
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:43559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:01.636039 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:01.636245 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:01.636281 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:01.637398 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:01.637460 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:01.637504 11790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:15:01.638896 11790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:01.639146 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:01.639376 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:01.644137 11798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:01.644124 11796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:01.644357 11790 server_base.cc:1034] running on GCE node
W20260924 00:15:01.644120 11795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:01.644574 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:01.644794 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:01.645926 11790 hybrid_clock.cc:648] HybridClock initialized: now 1790208901645911 us; error 31 us; skew 500 ppm
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:01.648339 11790 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:15:01.648753 11790 webserver.cc:460] Webserver started at http://127.1.44.131:35203/ using document root <none> and password file <none>
I20260924 00:15:01.649008 11790 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:01.649053 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:01.649209 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:15:01.650178 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "d79fed18032aecacf1a0191f86ce89fb"
server_key_iv: "176fd5b15831e752d905acf8d1684df0"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.650517 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "d79fed18032aecacf1a0191f86ce89fb"
server_key_iv: "176fd5b15831e752d905acf8d1684df0"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.651822 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:15:01.652519 11805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.652684 11790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:15:01.652766 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28"
format_stamp: "Formatted at 2026-09-24 00:15:01 on dist-test-slave-cm7b"
server_key: "d79fed18032aecacf1a0191f86ce89fb"
server_key_iv: "176fd5b15831e752d905acf8d1684df0"
server_key_version: "encryptionkey@0"
I20260924 00:15:01.652835 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:01.678663 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:01.680464 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:01.680604 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:01.680801 11790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:15:01.681126 11790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:15:01.681186 11790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.681226 11790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:15:01.681253 11790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:01.687220 11790 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:39223
I20260924 00:15:01.687259 11918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:39223 every 8 connection(s)
I20260924 00:15:01.687542 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:15:01.692806 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 11790
I20260924 00:15:01.692912 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:15:01.693068 1202 external_mini_cluster.cc:1428] Setting key fdb5c7322900c686db8a3335ace4a3d1
Sep 24 00:15:01 dist-test-slave-cm7b krb5kdc[11415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:01.697569 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.688169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:40841 (local address 127.1.44.190:44457)
0924 00:15:01.688224 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.688226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.688408 (+ 182us) server_negotiation.cc:407] Connection header received
0924 00:15:01.688850 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.688851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.688859 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.688901 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:01.689768 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.690259 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.691284 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.691472 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.691534 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.694105 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:01.694107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:01.694107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:01.694115 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:01.695922 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.696450 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.696451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.696451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.696516 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.696814 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.696815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.696815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.696966 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:01.697032 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.697363 (+ 331us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.697447 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:15:01.697757 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.688155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44457 (local address 127.1.44.131:40841)
0924 00:15:01.688292 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.688329 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.688763 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.688949 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.688952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.689496 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:15:01.689685 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.689689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.690344 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.690345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.691142 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.691144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.691547 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.691548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.691648 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.692190 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:15:01.692201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:15:01.694009 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:15:01.696004 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:01.696036 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:01.696038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:01.696365 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:01.696608 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:01.696609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:01.696609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:01.696744 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:01.697043 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:15:01.697057 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:15:01.697210 (+ 153us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.697348 (+ 138us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.697508 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260924 00:15:01.698315 11919 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44457
I20260924 00:15:01.698426 11919 heartbeater.cc:464] Registering TS with master...
I20260924 00:15:01.698596 11919 heartbeater.cc:511] Master 127.1.44.190:44457 requested a full tablet report, sending...
I20260924 00:15:01.698980 11460 ts_manager.cc:194] Registered new tserver with Master: 9b1a16d97ed2453cb1e0ae9f42db8f28 (127.1.44.131:39223)
I20260924 00:15:01.699527 11460 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:40841
I20260924 00:15:01.705019 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:15:01.711689 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.706251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.1.44.190:44457)
0924 00:15:01.706285 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.706286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.706395 (+ 109us) server_negotiation.cc:407] Connection header received
0924 00:15:01.706443 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.706443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.706450 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.706496 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:01.706954 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.707463 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.708128 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.708296 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.708350 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.709385 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:01.709387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:01.709387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:01.709392 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:01.710713 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.711034 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.711035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.711035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.711079 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:01.711280 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:01.711281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:01.711281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:01.711406 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:01.711477 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.711570 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.711624 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:15:01.713140 11460 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:15:01.714567 11460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:15:01.720737 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.716482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37591 (local address 127.0.0.1:45592)
0924 00:15:01.716725 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.716737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.716802 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.717132 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.717132 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.717135 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:01.717178 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.717180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.718162 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.718163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.719494 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.719496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.720439 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.720441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.720523 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.720536 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.720617 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.720666 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":25,"thread_start_us":47,"threads_started":1}
I20260924 00:15:01.720755 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.716814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.1.44.129:37591)
0924 00:15:01.716965 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.716971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.716980 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:15:01.717021 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.717022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.717035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.717121 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:01.717311 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.718063 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.719586 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.720302 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.720390 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.720452 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.720639 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.720680 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260924 00:15:01.721526 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.716473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43767 (local address 127.0.0.1:35420)
0924 00:15:01.716699 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.716737 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.716803 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.717073 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.717077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.717079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:01.717137 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.717138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.719251 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.719252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.720370 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.720372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.721255 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.721256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.721344 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.721347 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.721428 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.721464 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":46,"threads_started":1}
I20260924 00:15:01.721554 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.716569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35420 (local address 127.1.44.130:43767)
0924 00:15:01.716732 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.716737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.716794 (+ 57us) server_negotiation.cc:407] Connection header received
0924 00:15:01.716882 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.716884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.716901 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.716985 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:01.717217 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.718319 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.720448 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.721183 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.721256 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.721313 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.721445 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.721484 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260924 00:15:01.721602 11585 tablet_service.cc:1467] Processing CreateTablet for tablet e5cd7e2914c344bcbe5b5db389c5c986 (DEFAULT_TABLE table=test-table [id=a740a3ecd4ca40629d8bc842ea5bb27b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:15:01.721984 11585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cd7e2914c344bcbe5b5db389c5c986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:01.722184 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.716512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:39223 (local address 127.0.0.1:42474)
0924 00:15:01.716892 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.716902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.716938 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.717182 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.717183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.717185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:01.717228 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.717229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.718421 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.718422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.719493 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.719496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.721353 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.721354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.721420 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.721424 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.722064 (+ 640us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.722112 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":49,"threads_started":1}
I20260924 00:15:01.722218 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.716649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.1.44.131:39223)
0924 00:15:01.716993 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.716997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.717007 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:15:01.717037 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.717039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.717053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.717124 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:01.717302 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.718333 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.720488 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.721194 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.721262 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.721311 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.722103 (+ 792us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.722146 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":76,"threads_started":1}
I20260924 00:15:01.722726 11853 tablet_service.cc:1467] Processing CreateTablet for tablet e5cd7e2914c344bcbe5b5db389c5c986 (DEFAULT_TABLE table=test-table [id=a740a3ecd4ca40629d8bc842ea5bb27b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:15:01.722988 11853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cd7e2914c344bcbe5b5db389c5c986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:01.724594 11933 tablet_bootstrap.cc:492] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: Bootstrap starting.
I20260924 00:15:01.725052 11719 tablet_service.cc:1467] Processing CreateTablet for tablet e5cd7e2914c344bcbe5b5db389c5c986 (DEFAULT_TABLE table=test-table [id=a740a3ecd4ca40629d8bc842ea5bb27b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:15:01.725234 11933 tablet_bootstrap.cc:654] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:01.725353 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5cd7e2914c344bcbe5b5db389c5c986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:01.725777 11933 log.cc:824] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:01.726485 11933 tablet_bootstrap.cc:492] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: No bootstrap required, opened a new log
I20260924 00:15:01.726581 11933 ts_tablet_manager.cc:1397] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:15:01.727415 11934 tablet_bootstrap.cc:492] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Bootstrap starting.
I20260924 00:15:01.727969 11934 tablet_bootstrap.cc:654] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:01.728470 11934 log.cc:824] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:01.728669 11936 tablet_bootstrap.cc:492] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Bootstrap starting.
I20260924 00:15:01.728639 11933 raft_consensus.cc:348] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.728802 11933 raft_consensus.cc:374] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:01.728845 11933 raft_consensus.cc:729] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7c82cd444447ebbb47aa2bbe753a42, State: Initialized, Role: FOLLOWER
I20260924 00:15:01.728955 11933 consensus_queue.cc:260] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.729249 11933 ts_tablet_manager.cc:1428] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:15:01.729249 11651 heartbeater.cc:503] Master 127.1.44.190:44457 was elected leader, sending a full tablet report...
I20260924 00:15:01.729293 11936 tablet_bootstrap.cc:654] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:01.729825 11934 tablet_bootstrap.cc:492] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: No bootstrap required, opened a new log
I20260924 00:15:01.729874 11934 ts_tablet_manager.cc:1397] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 00:15:01.729861 11936 log.cc:824] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:01.730641 11936 tablet_bootstrap.cc:492] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: No bootstrap required, opened a new log
I20260924 00:15:01.730710 11936 ts_tablet_manager.cc:1397] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:15:01.731897 11934 raft_consensus.cc:348] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.732053 11934 raft_consensus.cc:374] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:01.732086 11934 raft_consensus.cc:729] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b1a16d97ed2453cb1e0ae9f42db8f28, State: Initialized, Role: FOLLOWER
I20260924 00:15:01.732196 11934 consensus_queue.cc:260] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.732445 11934 ts_tablet_manager.cc:1428] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:15:01.732513 11919 heartbeater.cc:503] Master 127.1.44.190:44457 was elected leader, sending a full tablet report...
I20260924 00:15:01.732641 11936 raft_consensus.cc:348] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.732797 11936 raft_consensus.cc:374] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:01.732844 11936 raft_consensus.cc:729] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e843a309669744b8bef439e9faae5aaf, State: Initialized, Role: FOLLOWER
I20260924 00:15:01.732944 11936 consensus_queue.cc:260] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.733199 11936 ts_tablet_manager.cc:1428] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:15:01.733217 11785 heartbeater.cc:503] Master 127.1.44.190:44457 was elected leader, sending a full tablet report...
W20260924 00:15:01.776451 11786 tablet.cc:2369] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:15:01.884415 11652 tablet.cc:2369] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:15:01.938060 11920 tablet.cc:2369] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:15:01.993003 11938 raft_consensus.cc:482] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:15:01.993132 11938 raft_consensus.cc:504] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.993541 11938 leader_election.cc:288] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e843a309669744b8bef439e9faae5aaf (127.1.44.130:43767), 9b1a16d97ed2453cb1e0ae9f42db8f28 (127.1.44.131:39223)
I20260924 00:15:01.996488 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.993838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:42185 (local address 127.1.44.131:39223)
0924 00:15:01.993882 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.993885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.993952 (+ 67us) server_negotiation.cc:407] Connection header received
0924 00:15:01.994020 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.994021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.994031 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.994107 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:01.994266 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.994735 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.995771 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.996141 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.996223 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.996231 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.996377 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.996412 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:15:01.996488 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.993801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:54205 (local address 127.1.44.130:43767)
0924 00:15:01.993861 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:01.993863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:01.993953 (+ 90us) server_negotiation.cc:407] Connection header received
0924 00:15:01.994020 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:01.994021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:01.994034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:01.994101 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:01.994277 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.994727 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.995775 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.996144 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.996223 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.996231 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:01.996388 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 00:15:01.996421 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:15:01.996515 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.993748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:39223 (local address 127.1.44.129:42185)
0924 00:15:01.993874 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.993885 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.993960 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.994109 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.994109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.994112 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:01.994195 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.994197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.994816 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.994817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.995676 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.995678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.996221 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.996222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.996299 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.996302 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.996360 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.996399 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":23,"thread_start_us":58,"threads_started":1}
I20260924 00:15:01.996520 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:01.993733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43767 (local address 127.1.44.129:54205)
0924 00:15:01.993867 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:01.993877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:01.993963 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:01.994106 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:01.994106 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:01.994108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:01.994207 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.994209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.994808 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.994808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.995688 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:01.995691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:01.996222 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:01.996223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:01.996298 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:01.996301 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:15:01.996375 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:15:01.996415 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":53,"thread_start_us":61,"threads_started":1}
I20260924 00:15:01.996855 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cd7e2914c344bcbe5b5db389c5c986" candidate_uuid: "be7c82cd444447ebbb47aa2bbe753a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" is_pre_election: true
I20260924 00:15:01.997059 11873 raft_consensus.cc:2454] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be7c82cd444447ebbb47aa2bbe753a42 in term 0.
I20260924 00:15:01.997093 11739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cd7e2914c344bcbe5b5db389c5c986" candidate_uuid: "be7c82cd444447ebbb47aa2bbe753a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e843a309669744b8bef439e9faae5aaf" is_pre_election: true
I20260924 00:15:01.997256 11739 raft_consensus.cc:2454] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be7c82cd444447ebbb47aa2bbe753a42 in term 0.
I20260924 00:15:01.997360 11541 leader_election.cc:302] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b1a16d97ed2453cb1e0ae9f42db8f28, be7c82cd444447ebbb47aa2bbe753a42; no voters:
I20260924 00:15:01.997529 11938 raft_consensus.cc:2781] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:15:01.997599 11938 raft_consensus.cc:482] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:15:01.997634 11938 raft_consensus.cc:3037] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:01.998355 11938 raft_consensus.cc:504] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:01.998509 11938 leader_election.cc:288] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [CANDIDATE]: Term 1 election: Requested vote from peers e843a309669744b8bef439e9faae5aaf (127.1.44.130:43767), 9b1a16d97ed2453cb1e0ae9f42db8f28 (127.1.44.131:39223)
I20260924 00:15:01.998629 11739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cd7e2914c344bcbe5b5db389c5c986" candidate_uuid: "be7c82cd444447ebbb47aa2bbe753a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e843a309669744b8bef439e9faae5aaf"
I20260924 00:15:01.998705 11739 raft_consensus.cc:3037] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:01.998710 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5cd7e2914c344bcbe5b5db389c5c986" candidate_uuid: "be7c82cd444447ebbb47aa2bbe753a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28"
I20260924 00:15:01.998773 11873 raft_consensus.cc:3037] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:01.999629 11873 raft_consensus.cc:2454] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be7c82cd444447ebbb47aa2bbe753a42 in term 1.
I20260924 00:15:01.999629 11739 raft_consensus.cc:2454] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be7c82cd444447ebbb47aa2bbe753a42 in term 1.
I20260924 00:15:01.999806 11538 leader_election.cc:302] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be7c82cd444447ebbb47aa2bbe753a42, e843a309669744b8bef439e9faae5aaf; no voters:
I20260924 00:15:01.999912 11938 raft_consensus.cc:2781] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:15:02.000068 11938 raft_consensus.cc:686] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 1 LEADER]: Becoming Leader. State: Replica: be7c82cd444447ebbb47aa2bbe753a42, State: Running, Role: LEADER
I20260924 00:15:02.000164 11938 consensus_queue.cc:237] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } }
I20260924 00:15:02.000911 11460 catalog_manager.cc:5495] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 reported cstate change: term changed from 0 to 1, leader changed from <none> to be7c82cd444447ebbb47aa2bbe753a42 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "be7c82cd444447ebbb47aa2bbe753a42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be7c82cd444447ebbb47aa2bbe753a42" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 37591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260924 00:15:02.071812 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:02.069439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.1.44.129:37591)
0924 00:15:02.069506 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:02.069508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:02.069517 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:15:02.069557 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:02.069558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:02.069584 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:02.069655 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:15:02.069818 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:02.070291 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:02.071035 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:02.071251 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:02.071322 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:02.071486 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:15:02.071596 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:15:02.071688 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:02.071707 (+ 19us) server_negotiation.cc:299] Negotiation successful
0924 00:15:02.071725 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:15:02.073616 11739 raft_consensus.cc:1261] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 1 FOLLOWER]: Refusing update from remote peer be7c82cd444447ebbb47aa2bbe753a42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:15:02.073616 11873 raft_consensus.cc:1261] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 1 FOLLOWER]: Refusing update from remote peer be7c82cd444447ebbb47aa2bbe753a42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:15:02.074026 11944 consensus_queue.cc:1035] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e843a309669744b8bef439e9faae5aaf" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:15:02.074177 11938 consensus_queue.cc:1035] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b1a16d97ed2453cb1e0ae9f42db8f28" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:15:02.076574 11949 mvcc.cc:205] Tried to move back new op lower bound from 7332695662890471424 to 7332695662593163264. Current Snapshot: MvccSnapshot[applied={T|T < 7332695662890471424}]
I20260924 00:15:02.078614 11459 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:15:02.078711 11459 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:15:02.079378 11459 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2b55bf3bde547229819ec2ac0e20425: Sending DeleteTablet for 3 replicas of tablet e5cd7e2914c344bcbe5b5db389c5c986
I20260924 00:15:02.079662 11719 tablet_service.cc:1514] Processing DeleteTablet for tablet e5cd7e2914c344bcbe5b5db389c5c986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:15:02 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:35420
I20260924 00:15:02.079665 11585 tablet_service.cc:1514] Processing DeleteTablet for tablet e5cd7e2914c344bcbe5b5db389c5c986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:15:02 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45592
I20260924 00:15:02.079946 11957 tablet_replica.cc:330] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42: stopping tablet replica
I20260924 00:15:02.079996 11853 tablet_service.cc:1514] Processing DeleteTablet for tablet e5cd7e2914c344bcbe5b5db389c5c986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:15:02 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:42474
I20260924 00:15:02.080049 11957 raft_consensus.cc:2229] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:15:02.080157 11957 raft_consensus.cc:2258] T e5cd7e2914c344bcbe5b5db389c5c986 P be7c82cd444447ebbb47aa2bbe753a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:15:02.080211 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11522
I20260924 00:15:02.080341 11959 tablet_replica.cc:330] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: stopping tablet replica
I20260924 00:15:02.080695 11959 raft_consensus.cc:2229] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:15:02.080833 11959 raft_consensus.cc:2258] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:15:02.081080 11958 tablet_replica.cc:330] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: stopping tablet replica
I20260924 00:15:02.081173 11958 raft_consensus.cc:2229] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:15:02.081274 11958 raft_consensus.cc:2258] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:15:02.081415 11959 ts_tablet_manager.cc:1905] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:15:02.082190 11958 ts_tablet_manager.cc:1905] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:15:02.083451 11959 ts_tablet_manager.cc:1918] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:15:02.083560 11959 log.cc:1196] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/e5cd7e2914c344bcbe5b5db389c5c986
I20260924 00:15:02.083583 11958 ts_tablet_manager.cc:1918] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:15:02.083654 11958 log.cc:1196] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/e5cd7e2914c344bcbe5b5db389c5c986
I20260924 00:15:02.083920 11958 ts_tablet_manager.cc:1939] T e5cd7e2914c344bcbe5b5db389c5c986 P e843a309669744b8bef439e9faae5aaf: Deleting consensus metadata
I20260924 00:15:02.083988 11959 ts_tablet_manager.cc:1939] T e5cd7e2914c344bcbe5b5db389c5c986 P 9b1a16d97ed2453cb1e0ae9f42db8f28: Deleting consensus metadata
I20260924 00:15:02.084280 11444 catalog_manager.cc:4841] TS e843a309669744b8bef439e9faae5aaf (127.1.44.130:43767): tablet e5cd7e2914c344bcbe5b5db389c5c986 (table test-table [id=a740a3ecd4ca40629d8bc842ea5bb27b]) successfully deleted
I20260924 00:15:02.084362 11447 catalog_manager.cc:4841] TS 9b1a16d97ed2453cb1e0ae9f42db8f28 (127.1.44.131:39223): tablet e5cd7e2914c344bcbe5b5db389c5c986 (table test-table [id=a740a3ecd4ca40629d8bc842ea5bb27b]) successfully deleted
W20260924 00:15:02.086030 11446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:37591 (error 108)
I20260924 00:15:02.086277 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11656
I20260924 00:15:02.086452 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:02.086320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:37591 (local address 127.0.0.1:45612)
0924 00:15:02.086379 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:02.086416 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:37591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260924 00:15:02.086843 11446 catalog_manager.cc:4584] TS be7c82cd444447ebbb47aa2bbe753a42 (127.1.44.129:37591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5cd7e2914c344bcbe5b5db389c5c986: Network error: Client connection negotiation failed: client connection to 127.1.44.129:37591: connect: Connection refused (error 111)
I20260924 00:15:02.091099 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11790
I20260924 00:15:02.095561 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 11428
2026-09-24T00:15:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2183 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:15:02 dist-test-slave-cm7b krb5kdc[11963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:15:02 dist-test-slave-cm7b krb5kdc[11963](info): set up 2 sockets
Sep 24 00:15:02 dist-test-slave-cm7b krb5kdc[11963](info): commencing operation
krb5kdc: starting...
W20260924 00:15:03.120060 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:15:03 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:15:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/rangeradmin_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/rangeradmin_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/rangerlookup_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/rangerlookup_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
I20260924 00:15:03.270358 1202 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskoOH61c/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/postgres -l logfile start
2026-09-24 00:15:05.674 UTC [11993] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 00:15:05.674 UTC [11993] LOG: listening on IPv4 address "127.1.44.148", port 44593
2026-09-24 00:15:05.677 UTC [11993] LOG: listening on Unix socket "/tmp/.s.PGSQL.44593"
2026-09-24 00:15:05.698 UTC [11996] LOG: database system was shut down at 2026-09-24 00:15:04 UTC
2026-09-24 00:15:05.702 UTC [11993] LOG: database system is ready to accept connections
I20260924 00:15:06.652107 1202 mini_postgres.cc:96] Postgres bound to 44593
2026-09-24 00:15:06.657 UTC [12004] FATAL: database "slave" does not exist
127.1.44.148:44593 - accepting connections
I20260924 00:15:06.658545 1202 mini_ranger.cc:161] Starting Ranger...
I20260924 00:15:06.672554 1202 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 00:15:07.132390 1202 mini_ranger.cc:87] Created ranger Postgres database
I20260924 00:15:07.132488 1202 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-admin
2026-09-24 00:15:07,389 [I] DB FLAVOR :POSTGRES
2026-09-24 00:15:07,389 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:15:07,389 [I] Checking connection..
2026-09-24 00:15:07,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 00:15:07,648 [I] Checking connection passed.
2026-09-24 00:15:07,648 [I] --------- Verifying version history table ---------
2026-09-24 00:15:07,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:15:07,905 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:15:07,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:15:08,163 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:15:08,163 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 00:15:08,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 00:15:08.390 UTC [12068] WARNING: there is no transaction in progress
2026-09-24 00:15:08,395 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 00:15:08,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:15:08,647 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 00:15:08,648 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:15:08,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 00:15:08,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 00:15:09,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cm7b.c.gcp-upstream.internal','N') ;"
2026-09-24 00:15:09,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:15:09,556 [I] Table x_portal_user does not exist in database ranger
2026-09-24 00:15:09,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:15:09,807 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 00:15:09,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:15:10,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:15:10,288 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 00:15:10,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 00:15:11.194 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.210 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.226 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.240 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.255 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.272 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.334 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.340 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.354 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.364 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.373 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.386 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.396 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.402 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.409 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:11.415 UTC [12176] WARNING: there is no transaction in progress
2026-09-24 00:15:12,023 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 00:15:12,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 00:15:12,247 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 00:15:12,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:15:12,502 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 00:15:12,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:15:12,723 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 00:15:12,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:15:12,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:15:13,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cm7b.c.gcp-upstream.internal';"
2026-09-24 00:15:13,465 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 00:15:13,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 00:15:13,711 [I] DB_PATCHES have already been applied
I20260924 00:15:13.717027 1202 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-admin:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskoOH61c/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 00:15:13.717089 1202 mini_ranger.cc:193] Using host: 127.1.44.148
I20260924 00:15:13.719992 1202 mini_ranger.cc:227] Ranger admin URL: http://127.1.44.148:40271
Sep 24, 2026 12:15:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:15:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:15:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 12:15:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 12:15:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 12:15:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 12:15:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 00:15:14.827872 1202 mini_ranger.cc:160] Time spent starting Ranger: real 8.169s user 0.001s sys 0.007s
Sep 24, 2026 12:15:33 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 12:15:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 24, 2026 12:15:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 12:15:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 12:15:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 00:15:34.972452 1202 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/rangerkms_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/rangerkms_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.44.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.44.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 00:15:35 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208935, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 00:15:35.049516 1202 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 00:15:35.058094 1202 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 00:15:35.902043 1202 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 00:15:35.902132 1202 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms
I20260924 00:15:35.902154 1202 mini_ranger_kms.cc:227] Using postgres at 127.1.44.148:44593
2026-09-24 00:15:36,075 [I] DB FLAVOR :POSTGRES
2026-09-24 00:15:36,076 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:15:36,076 [I] Checking connection
2026-09-24 00:15:36,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 00:15:36,311 [I] connection success
2026-09-24 00:15:36,311 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 00:15:36,312 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 00:15:36,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 00:15:36,536 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 00:15:36,537 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:15:36,537 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 00:15:36,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44593/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 00:15:36,782 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 00:15:37.222666 1202 mini_ranger_kms.cc:310] Created kms service
I20260924 00:15:37.416842 1202 mini_ranger_kms.cc:326] Created kudu user
I20260924 00:15:37.458706 1202 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 00:15:37 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208935, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:15:37.852141 1202 mini_ranger_kms.cc:384] Added ranger policy
I20260924 00:15:37.852252 1202 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms:/tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 00:15:37.852275 1202 mini_ranger_kms.cc:240] Using host: 127.1.44.148
I20260924 00:15:37.855403 1202 mini_ranger_kms.cc:276] Ranger KMS PID: 12356
I20260924 00:15:37.855527 1202 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.44.148:33691
Sep 24, 2026 12:15:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:15:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:15:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 12:15:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 12:15:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 12:15:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 24 00:15:39 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790208939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.44.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:40.088493 1202 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.039s user 0.000s sys 0.155s
I20260924 00:15:40.088600 1202 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 00:15:40.088691 1202 mini_ranger_kms.cc:401] 127.1.44.148:33691/kms/v1/keys
Sep 24 00:15:40 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.44.148@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
Sep 24 00:15:40 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208939, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.44.148@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
Sep 24 00:15:42 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:15:42.749272 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42643
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:42643
--ranger_config_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:33691/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:42.827955 12407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:42.828130 12407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:42.828150 12407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:42.829265 12407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:15:42.829306 12407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:42.829320 12407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:42.829332 12407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:15:42.829383 12407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:15:42.830809 12407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.44.148:33691/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:42643
--ranger_config_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:42.831027 12407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:42.831285 12407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:42.836122 12413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:42.836071 12412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:42.836098 12407 server_base.cc:1034] running on GCE node
W20260924 00:15:42.836071 12415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:42.836520 12407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:42.836782 12407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:42.837942 12407 hybrid_clock.cc:648] HybridClock initialized: now 1790208942837925 us; error 48 us; skew 500 ppm
Sep 24 00:15:42 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:42.842949 12407 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:15:42.843430 12407 webserver.cc:460] Webserver started at http://127.1.44.190:42891/ using document root <none> and password file <none>
I20260924 00:15:42.843636 12407 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:42.843695 12407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:42.843844 12407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:15:42 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208942, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:15:42.893412 12407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "7faa049e97234d089a0ec8fa49ad78b9"
format_stamp: "Formatted at 2026-09-24 00:15:42 on dist-test-slave-cm7b"
server_key: "b802e6730ae0280c78d09cc2e0fb4178"
server_key_iv: "601a392d3312e2ce213ffda509b80c2c"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:42.893870 12407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "7faa049e97234d089a0ec8fa49ad78b9"
format_stamp: "Formatted at 2026-09-24 00:15:42 on dist-test-slave-cm7b"
server_key: "b802e6730ae0280c78d09cc2e0fb4178"
server_key_iv: "601a392d3312e2ce213ffda509b80c2c"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:42.895619 12407 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 00:15:42.906067 12422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:42.906263 12407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:15:42.906358 12407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "7faa049e97234d089a0ec8fa49ad78b9"
format_stamp: "Formatted at 2026-09-24 00:15:42 on dist-test-slave-cm7b"
server_key: "b802e6730ae0280c78d09cc2e0fb4178"
server_key_iv: "601a392d3312e2ce213ffda509b80c2c"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:42.906433 12407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:42.919843 12407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:42.921674 12407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:42.921826 12407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:42.926420 12407 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:42643
I20260924 00:15:42.926636 12474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:42643 every 8 connection(s)
I20260924 00:15:42.926856 12407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:15:42.927520 12475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:42.929600 12475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9: Bootstrap starting.
I20260924 00:15:42.930526 12475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:42.930800 12475 log.cc:824] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:42.931502 12475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9: No bootstrap required, opened a new log
I20260924 00:15:42.933415 12475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7faa049e97234d089a0ec8fa49ad78b9" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42643 } }
I20260924 00:15:42.933574 12475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:42.933621 12475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7faa049e97234d089a0ec8fa49ad78b9, State: Initialized, Role: FOLLOWER
I20260924 00:15:42.933794 12475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7faa049e97234d089a0ec8fa49ad78b9" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42643 } }
I20260924 00:15:42.933889 12475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:15:42.933933 12475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:15:42.933980 12475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:42.934285 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 12407
I20260924 00:15:42.934386 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:15:42.934532 12475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7faa049e97234d089a0ec8fa49ad78b9" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42643 } }
I20260924 00:15:42.934582 1202 external_mini_cluster.cc:1428] Setting key 9228cc5920ca022652fab6e8cad16b52
I20260924 00:15:42.934604 12475 leader_election.cc:302] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7faa049e97234d089a0ec8fa49ad78b9; no voters:
I20260924 00:15:42.934742 12475 leader_election.cc:288] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:15:42.934769 12478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:15:42.934970 12475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:15:42.935123 12478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [term 1 LEADER]: Becoming Leader. State: Replica: 7faa049e97234d089a0ec8fa49ad78b9, State: Running, Role: LEADER
I20260924 00:15:42.935313 12478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7faa049e97234d089a0ec8fa49ad78b9" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42643 } }
I20260924 00:15:42.935710 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7faa049e97234d089a0ec8fa49ad78b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7faa049e97234d089a0ec8fa49ad78b9" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42643 } } }
I20260924 00:15:42.935858 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [sys.catalog]: This master's current role is: LEADER
I20260924 00:15:42.936036 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7faa049e97234d089a0ec8fa49ad78b9. Latest consensus state: current_term: 1 leader_uuid: "7faa049e97234d089a0ec8fa49ad78b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7faa049e97234d089a0ec8fa49ad78b9" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42643 } } }
I20260924 00:15:42.936146 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9 [sys.catalog]: This master's current role is: LEADER
I20260924 00:15:42.936328 12475 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 00:15:42.939098 12492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:15:42.940201 12492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:15:42.945420 12492 catalog_manager.cc:1347] Generated new cluster ID: f4abfd79196a4188aeb31f38510baf1e
I20260924 00:15:42.945463 12492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:15:42 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:42.960528 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:42.934963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.1.44.190:42643)
0924 00:15:42.935426 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:42.935436 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:42.935466 (+ 30us) server_negotiation.cc:407] Connection header received
0924 00:15:42.935575 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:42.935582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:42.935634 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:42.935775 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:42.936590 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:42.941388 (+ 4798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:42.942539 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:42.943225 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:42.943344 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:42.948762 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:42.948766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:42.948766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:42.948779 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:42.950816 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:42.959258 (+ 8442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:42.959261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:42.959261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:42.959366 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:42.959589 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:42.959590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:42.959590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:42.959888 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:42.960032 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:42.960159 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 00:15:42.960288 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":61,"threads_started":1}
I20260924 00:15:42.970466 12492 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:15:42.971266 12492 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:15:42.977510 12492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9: Generated new TSK 0
I20260924 00:15:42.977716 12492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:15:43.675637 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:33691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42643
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:43.779583 12520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:43.779838 12520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:43.779870 12520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:43.781464 12520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:43.781514 12520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:43.781550 12520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:15:43.782999 12520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.44.148:33691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:43.783237 12520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:43.783478 12520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:43.788226 12528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:43.788235 12526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:43.788452 12520 server_base.cc:1034] running on GCE node
W20260924 00:15:43.788225 12525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:43.788735 12520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:43.789000 12520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:43.790177 12520 hybrid_clock.cc:648] HybridClock initialized: now 1790208943790161 us; error 32 us; skew 500 ppm
Sep 24 00:15:43 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208943, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:43.793100 12520 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:15:43.793516 12520 webserver.cc:460] Webserver started at http://127.1.44.129:38581/ using document root <none> and password file <none>
I20260924 00:15:43.793730 12520 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:43.793795 12520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:43.793941 12520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:15:43 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208943, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:15:43.810317 12520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "1c19057b4f19476c89b2fdacd864e4cf"
format_stamp: "Formatted at 2026-09-24 00:15:43 on dist-test-slave-cm7b"
server_key: "0c2fea14177f93a6390acf7e6cbebf95"
server_key_iv: "87de5cd5fcb688c33e98a23edef89f04"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:43.810786 12520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "1c19057b4f19476c89b2fdacd864e4cf"
format_stamp: "Formatted at 2026-09-24 00:15:43 on dist-test-slave-cm7b"
server_key: "0c2fea14177f93a6390acf7e6cbebf95"
server_key_iv: "87de5cd5fcb688c33e98a23edef89f04"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:43.816913 12520 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260924 00:15:43.829493 12535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:43.829691 12520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:15:43.829770 12520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "1c19057b4f19476c89b2fdacd864e4cf"
format_stamp: "Formatted at 2026-09-24 00:15:43 on dist-test-slave-cm7b"
server_key: "0c2fea14177f93a6390acf7e6cbebf95"
server_key_iv: "87de5cd5fcb688c33e98a23edef89f04"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:43.829838 12520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:43.841214 12520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:43.843492 12520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:43.843657 12520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:43.843917 12520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:15:43.844287 12520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:15:43.844339 12520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:43.844369 12520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:15:43.844388 12520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:43.852133 12520 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:45271
I20260924 00:15:43.852217 12650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:45271 every 8 connection(s)
I20260924 00:15:43.852754 12520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:15:43 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208943, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:43.860525 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 12520
I20260924 00:15:43.860633 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:15:43.860853 1202 external_mini_cluster.cc:1428] Setting key 2605c03e3d55b98c1320e554469495bf
I20260924 00:15:43.864117 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:43.853443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42643 (local address 127.1.44.129:38733)
0924 00:15:43.853605 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:43.853650 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:43.854300 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:43.854677 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:43.854681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:43.854900 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:15:43.855147 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:43.855152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:43.856250 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:43.856251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:43.857487 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:43.857490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:43.858022 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:43.858023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:43.858155 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:43.858731 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:15:43.858742 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:15:43.860325 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:15:43.862524 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:43.862530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:43.862532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:43.862819 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:43.863060 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:43.863061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:43.863062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:43.863179 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:43.863486 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:15:43.863492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:15:43.863646 (+ 154us) client_negotiation.cc:769] Sending connection context
0924 00:15:43.863743 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 00:15:43.863881 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260924 00:15:43.864532 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:43.853464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:38733 (local address 127.1.44.190:42643)
0924 00:15:43.853601 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:43.853603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:43.853748 (+ 145us) server_negotiation.cc:407] Connection header received
0924 00:15:43.854470 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:43.854471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:43.854487 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:43.854554 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:43.855319 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:43.856129 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:43.857585 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:43.857899 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:43.857960 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:43.860430 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:43.860432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:43.860433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:43.860441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:43.862428 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:43.862924 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:43.862926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:43.862926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:43.862991 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:43.863271 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:43.863272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:43.863272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:43.863399 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:43.863459 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:43.864295 (+ 836us) server_negotiation.cc:299] Negotiation successful
0924 00:15:43.864383 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260924 00:15:43.865011 12651 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42643
I20260924 00:15:43.865106 12651 heartbeater.cc:464] Registering TS with master...
I20260924 00:15:43.865283 12651 heartbeater.cc:511] Master 127.1.44.190:42643 requested a full tablet report, sending...
I20260924 00:15:43.865793 12439 ts_manager.cc:194] Registered new tserver with Master: 1c19057b4f19476c89b2fdacd864e4cf (127.1.44.129:45271)
I20260924 00:15:43.866575 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:38733
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:15:43.890527 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:33691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42643
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:44.000968 12657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:44.001219 12657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:44.001251 12657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:44.002873 12657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:44.002928 12657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:44.002964 12657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:15:44.005040 12657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.44.148:33691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:44.005282 12657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:44.005607 12657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:44.011638 12667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:44.011780 12664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:44.011647 12665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:44.012903 12657 server_base.cc:1034] running on GCE node
I20260924 00:15:44.013123 12657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:44.013358 12657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:44.014524 12657 hybrid_clock.cc:648] HybridClock initialized: now 1790208944014506 us; error 32 us; skew 500 ppm
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:44.017642 12657 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:15:44.018165 12657 webserver.cc:460] Webserver started at http://127.1.44.130:38087/ using document root <none> and password file <none>
I20260924 00:15:44.018481 12657 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:44.018558 12657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:44.018695 12657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:44.032456 12657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a"
format_stamp: "Formatted at 2026-09-24 00:15:44 on dist-test-slave-cm7b"
server_key: "16feeb337493ffdf77e043b5e27782eb"
server_key_iv: "e0cdc612c4781608b84a47f58d0ede72"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:44.032939 12657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a"
format_stamp: "Formatted at 2026-09-24 00:15:44 on dist-test-slave-cm7b"
server_key: "16feeb337493ffdf77e043b5e27782eb"
server_key_iv: "e0cdc612c4781608b84a47f58d0ede72"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:44.034962 12657 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:15:44.053660 12674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:44.053874 12657 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260924 00:15:44.053972 12657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a"
format_stamp: "Formatted at 2026-09-24 00:15:44 on dist-test-slave-cm7b"
server_key: "16feeb337493ffdf77e043b5e27782eb"
server_key_iv: "e0cdc612c4781608b84a47f58d0ede72"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:44.054049 12657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:44.090358 12657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:44.092700 12657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:44.092883 12657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:44.093142 12657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:15:44.093515 12657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:15:44.093595 12657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:44.093654 12657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:15:44.093703 12657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:44.100607 12657 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:44085
I20260924 00:15:44.101069 12657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:15:44.102459 12787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:44085 every 8 connection(s)
I20260924 00:15:44.108263 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 12657
I20260924 00:15:44.108359 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:15:44.108546 1202 external_mini_cluster.cc:1428] Setting key 3cd4c1195eb9d5f55dca699fc85da8c1
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:44.113003 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.101807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42643 (local address 127.1.44.130:54887)
0924 00:15:44.102792 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.102834 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.103495 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.103861 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.103865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.104091 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:15:44.104345 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.104352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.105332 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.105333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.106587 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.106590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.106978 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.106979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.107099 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.107626 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:15:44.107641 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:15:44.109063 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:15:44.111044 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:44.111051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:44.111053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:44.111378 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:44.111594 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:44.111595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:44.111595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:44.111710 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:44.112386 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:15:44.112392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:15:44.112537 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.112629 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.112769 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":95,"threads_started":1}
I20260924 00:15:44.113425 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.101799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:54887 (local address 127.1.44.190:42643)
0924 00:15:44.101849 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.101851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.102928 (+ 1077us) server_negotiation.cc:407] Connection header received
0924 00:15:44.103646 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.103647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.103661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.103692 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:44.104463 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.105212 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.106681 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.106877 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.106972 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.109169 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:44.109170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:44.109171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:44.109178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:44.110953 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:44.111468 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:44.111469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:44.111470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:44.111527 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:44.112169 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:44.112170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:44.112170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:44.112305 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:44.112361 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.113228 (+ 867us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.113312 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:15:44.113729 12788 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42643
I20260924 00:15:44.113830 12788 heartbeater.cc:464] Registering TS with master...
I20260924 00:15:44.114017 12788 heartbeater.cc:511] Master 127.1.44.190:42643 requested a full tablet report, sending...
I20260924 00:15:44.114375 12439 ts_manager.cc:194] Registered new tserver with Master: c8bbbe202bb24d7e81a90c0ce1e50d3a (127.1.44.130:44085)
I20260924 00:15:44.114840 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:54887
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:15:44.142459 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:33691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42643
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:15:44.256289 12794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:15:44.256485 12794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:15:44.256506 12794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:15:44.257637 12794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:15:44.257678 12794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:15:44.257702 12794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:15:44.259260 12794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.44.148:33691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:15:44.259482 12794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:15:44.259725 12794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:15:44.264395 12802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:44.264616 12800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:15:44.264779 12799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:15:44.264902 12794 server_base.cc:1034] running on GCE node
I20260924 00:15:44.265079 12794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:15:44.265321 12794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:15:44.266495 12794 hybrid_clock.cc:648] HybridClock initialized: now 1790208944266475 us; error 32 us; skew 500 ppm
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:15:44.269338 12794 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:15:44.269742 12794 webserver.cc:460] Webserver started at http://127.1.44.131:38241/ using document root <none> and password file <none>
I20260924 00:15:44.269980 12794 fs_manager.cc:359] Metadata directory not provided
I20260924 00:15:44.270049 12794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:15:44.270196 12794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:15:44.297117 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "63b45adfc8cf4f7b8ff7889d6da1b532"
format_stamp: "Formatted at 2026-09-24 00:15:44 on dist-test-slave-cm7b"
server_key: "357bed9746fc15a4983318f2ae03f47d"
server_key_iv: "d9ac6e268a7467baf7b34a5d41660237"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:44.297567 12794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "63b45adfc8cf4f7b8ff7889d6da1b532"
format_stamp: "Formatted at 2026-09-24 00:15:44 on dist-test-slave-cm7b"
server_key: "357bed9746fc15a4983318f2ae03f47d"
server_key_iv: "d9ac6e268a7467baf7b34a5d41660237"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:44.299396 12794 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:15:44.314005 12809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:44.314222 12794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:15:44.314301 12794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "63b45adfc8cf4f7b8ff7889d6da1b532"
format_stamp: "Formatted at 2026-09-24 00:15:44 on dist-test-slave-cm7b"
server_key: "357bed9746fc15a4983318f2ae03f47d"
server_key_iv: "d9ac6e268a7467baf7b34a5d41660237"
server_key_version: "kuduclusterkey@0"
I20260924 00:15:44.314383 12794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:15:44.332767 12794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:15:44.335460 12794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:15:44.335610 12794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:15:44.335812 12794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:15:44.336238 12794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:15:44.336315 12794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:44.336373 12794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:15:44.336419 12794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:15:44.343943 12794 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:43277
I20260924 00:15:44.344033 12922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:43277 every 8 connection(s)
I20260924 00:15:44.344388 12794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:15:44.351878 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 12794
I20260924 00:15:44.351997 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:15:44.352177 1202 external_mini_cluster.cc:1428] Setting key 1f51c7bd6cd63f8eb21932d88429de57
I20260924 00:15:44.355419 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.345466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:48673 (local address 127.1.44.190:42643)
0924 00:15:44.345508 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.345511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.345522 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:15:44.346161 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.346162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.346176 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.346207 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:44.346972 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.347823 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.349297 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.349609 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.349695 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.352173 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:44.352174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:44.352175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:44.352182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:44.353921 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:44.354376 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:44.354378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:44.354378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:44.354434 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:44.354695 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:44.354696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:44.354696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:44.354820 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:44.354860 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.355247 (+ 387us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.355327 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:15:44.355561 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.345201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42643 (local address 127.1.44.131:48673)
0924 00:15:44.345373 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.345419 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.346029 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.346379 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.346383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.346603 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:15:44.346846 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.346852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.347951 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.347952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.349198 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.349201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.349708 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.349709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.349847 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.350386 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:15:44.350396 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:15:44.352060 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:15:44.354013 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:44.354020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:44.354022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:44.354287 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:44.354504 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:15:44.354505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:15:44.354505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:15:44.354622 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:15:44.354919 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:15:44.354925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:15:44.355068 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.355175 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.355345 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260924 00:15:44.355957 12923 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42643
I20260924 00:15:44.356042 12923 heartbeater.cc:464] Registering TS with master...
I20260924 00:15:44.356209 12923 heartbeater.cc:511] Master 127.1.44.190:42643 requested a full tablet report, sending...
I20260924 00:15:44.356600 12439 ts_manager.cc:194] Registered new tserver with Master: 63b45adfc8cf4f7b8ff7889d6da1b532 (127.1.44.131:43277)
I20260924 00:15:44.357096 12439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:48673
I20260924 00:15:44.363739 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:15:44.373276 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.365314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.1.44.190:42643)
0924 00:15:44.365349 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.365351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.365616 (+ 265us) server_negotiation.cc:407] Connection header received
0924 00:15:44.365711 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.365712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.365721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.365753 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:15:44.366485 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.367211 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.368296 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.368595 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.368642 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.370228 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:15:44.370229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:15:44.370230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:15:44.370236 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:15:44.371958 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:44.372365 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:44.372366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:44.372366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:44.372427 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:15:44.372676 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:15:44.372677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:15:44.372677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:15:44.372802 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:15:44.372891 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.373117 (+ 226us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.373190 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:15:44.375540 12439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:15:44.378005 12439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:15:44.386019 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.381171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44085 (local address 127.0.0.1:34112)
0924 00:15:44.381338 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.381378 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.381442 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.382000 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.382003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.382005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.382062 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.382064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.383419 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.383420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.384724 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.384727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.385740 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.385741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.385824 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.385839 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.385902 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.385947 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260924 00:15:44.386713 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.381619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.1.44.130:44085)
0924 00:15:44.381799 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.381803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.381814 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:15:44.381853 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.381856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.381883 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.382142 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.382211 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.383319 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.384854 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.385636 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.386488 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.386571 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.386598 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.386636 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260924 00:15:44.387518 12704 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca81e50d6044d7590f7cbd9e2727356 (DEFAULT_TABLE table=test-table [id=02a64f4ea9f4420bb02af42d73d65931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:15:44.387881 12704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca81e50d6044d7590f7cbd9e2727356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:44.391865 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.381893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:45271 (local address 127.0.0.1:45710)
0924 00:15:44.383300 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.383311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.383360 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.384564 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.384565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.384567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.384620 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.384621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.387130 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.387131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.388274 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.388277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.391627 (+ 3350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.391628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.391704 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.391708 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.391758 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.391796 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1390,"thread_start_us":88,"threads_started":1}
I20260924 00:15:44.392194 12942 tablet_bootstrap.cc:492] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Bootstrap starting.
I20260924 00:15:44.392334 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.381412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:43277 (local address 127.0.0.1:60770)
0924 00:15:44.382508 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.382517 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.382562 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.383665 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.383666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.383668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.383707 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.383708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.388393 (+ 4685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.388394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.389741 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.389744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.392139 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.392140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.392203 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.392207 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.392236 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.392268 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"thread_start_us":64,"threads_started":1}
I20260924 00:15:44.392980 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.381682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.1.44.131:43277)
0924 00:15:44.383467 (+ 1785us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.383471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.383480 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:15:44.383514 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.383516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.383531 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.383600 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.383775 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.386049 (+ 2274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.389859 (+ 3810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.390611 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.392747 (+ 2136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.392827 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.392853 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.392889 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":74,"threads_started":1}
I20260924 00:15:44.393415 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.382100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.1.44.129:45271)
0924 00:15:44.384368 (+ 2268us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.384373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.384382 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:15:44.384413 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.384415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.384430 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.384501 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.384689 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.387036 (+ 2347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.390699 (+ 3663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.391467 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.391540 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.391592 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.393293 (+ 1701us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.393332 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2236,"thread_start_us":59,"threads_started":1}
I20260924 00:15:44.393625 12857 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca81e50d6044d7590f7cbd9e2727356 (DEFAULT_TABLE table=test-table [id=02a64f4ea9f4420bb02af42d73d65931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:15:44.393941 12857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca81e50d6044d7590f7cbd9e2727356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:44.394024 12585 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca81e50d6044d7590f7cbd9e2727356 (DEFAULT_TABLE table=test-table [id=02a64f4ea9f4420bb02af42d73d65931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:15:44.394308 12585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca81e50d6044d7590f7cbd9e2727356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:15:44.397337 12943 tablet_bootstrap.cc:492] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Bootstrap starting.
I20260924 00:15:44.397457 12942 tablet_bootstrap.cc:654] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:44.397832 12942 log.cc:824] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:44.398413 12943 tablet_bootstrap.cc:654] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:44.398630 12942 tablet_bootstrap.cc:492] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: No bootstrap required, opened a new log
I20260924 00:15:44.398693 12942 ts_tablet_manager.cc:1397] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260924 00:15:44.398761 12943 log.cc:824] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:44.399030 12944 tablet_bootstrap.cc:492] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: Bootstrap starting.
I20260924 00:15:44.399665 12944 tablet_bootstrap.cc:654] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: Neither blocks nor log segments found. Creating new log.
I20260924 00:15:44.399919 12944 log.cc:824] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: Log is configured to *not* fsync() on all Append() calls
I20260924 00:15:44.400588 12943 tablet_bootstrap.cc:492] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: No bootstrap required, opened a new log
I20260924 00:15:44.400638 12943 ts_tablet_manager.cc:1397] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:15:44.400789 12942 raft_consensus.cc:348] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.400950 12942 raft_consensus.cc:374] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:44.400981 12942 raft_consensus.cc:729] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8bbbe202bb24d7e81a90c0ce1e50d3a, State: Initialized, Role: FOLLOWER
I20260924 00:15:44.401088 12942 consensus_queue.cc:260] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.401320 12942 ts_tablet_manager.cc:1428] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:15:44.401657 12788 heartbeater.cc:503] Master 127.1.44.190:42643 was elected leader, sending a full tablet report...
I20260924 00:15:44.403234 12943 raft_consensus.cc:348] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.403439 12943 raft_consensus.cc:374] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:44.403517 12943 raft_consensus.cc:729] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b45adfc8cf4f7b8ff7889d6da1b532, State: Initialized, Role: FOLLOWER
I20260924 00:15:44.403672 12943 consensus_queue.cc:260] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.403931 12943 ts_tablet_manager.cc:1428] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:15:44.404062 12923 heartbeater.cc:503] Master 127.1.44.190:42643 was elected leader, sending a full tablet report...
I20260924 00:15:44.405089 12944 tablet_bootstrap.cc:492] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: No bootstrap required, opened a new log
I20260924 00:15:44.405174 12944 ts_tablet_manager.cc:1397] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260924 00:15:44.406874 12944 raft_consensus.cc:348] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.407014 12944 raft_consensus.cc:374] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:15:44.407087 12944 raft_consensus.cc:729] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c19057b4f19476c89b2fdacd864e4cf, State: Initialized, Role: FOLLOWER
I20260924 00:15:44.407228 12944 consensus_queue.cc:260] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.407500 12651 heartbeater.cc:503] Master 127.1.44.190:42643 was elected leader, sending a full tablet report...
I20260924 00:15:44.407721 12944 ts_tablet_manager.cc:1428] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
W20260924 00:15:44.595290 12924 tablet.cc:2369] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:15:44.603629 12652 tablet.cc:2369] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:15:44.606181 12789 tablet.cc:2369] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 00:15:44 dist-test-slave-cm7b krb5kdc[11963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:15:44.860818 12950 raft_consensus.cc:482] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:15:44.860952 12950 raft_consensus.cc:504] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.861308 12950 leader_election.cc:288] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8bbbe202bb24d7e81a90c0ce1e50d3a (127.1.44.130:44085), 63b45adfc8cf4f7b8ff7889d6da1b532 (127.1.44.131:43277)
I20260924 00:15:44.862020 12949 raft_consensus.cc:482] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:15:44.862128 12949 raft_consensus.cc:504] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.862466 12949 leader_election.cc:288] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8bbbe202bb24d7e81a90c0ce1e50d3a (127.1.44.130:44085), 1c19057b4f19476c89b2fdacd864e4cf (127.1.44.129:45271)
I20260924 00:15:44.866434 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.861932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:53457 (local address 127.1.44.131:43277)
0924 00:15:44.861986 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.861988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.862080 (+ 92us) server_negotiation.cc:407] Connection header received
0924 00:15:44.862172 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.862173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.862181 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.862210 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.863445 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.864153 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.865527 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.866058 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.866109 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.866118 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.866332 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.866371 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:15:44.866680 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.861732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:43277 (local address 127.1.44.129:53457)
0924 00:15:44.862024 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.862033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.862101 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.862269 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.862270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.862271 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.863348 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.863352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.864251 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.864252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.865421 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.865423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.866178 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.866179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.866249 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.866252 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.866557 (+ 305us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.866595 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":993,"thread_start_us":74,"threads_started":1}
I20260924 00:15:44.866974 12877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca81e50d6044d7590f7cbd9e2727356" candidate_uuid: "1c19057b4f19476c89b2fdacd864e4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" is_pre_election: true
I20260924 00:15:44.867096 12877 raft_consensus.cc:2454] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c19057b4f19476c89b2fdacd864e4cf in term 0.
I20260924 00:15:44.867317 12541 leader_election.cc:302] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c19057b4f19476c89b2fdacd864e4cf, 63b45adfc8cf4f7b8ff7889d6da1b532; no voters:
I20260924 00:15:44.867460 12950 raft_consensus.cc:2781] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:15:44.867515 12950 raft_consensus.cc:482] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:15:44.867543 12950 raft_consensus.cc:3037] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:44.868431 12950 raft_consensus.cc:504] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.868546 12950 leader_election.cc:288] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [CANDIDATE]: Term 1 election: Requested vote from peers c8bbbe202bb24d7e81a90c0ce1e50d3a (127.1.44.130:44085), 63b45adfc8cf4f7b8ff7889d6da1b532 (127.1.44.131:43277)
I20260924 00:15:44.868726 12877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca81e50d6044d7590f7cbd9e2727356" candidate_uuid: "1c19057b4f19476c89b2fdacd864e4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532"
I20260924 00:15:44.868785 12877 raft_consensus.cc:3037] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:44.869549 12877 raft_consensus.cc:2454] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c19057b4f19476c89b2fdacd864e4cf in term 1.
I20260924 00:15:44.870494 12541 leader_election.cc:302] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c19057b4f19476c89b2fdacd864e4cf, 63b45adfc8cf4f7b8ff7889d6da1b532; no voters:
I20260924 00:15:44.870708 12950 raft_consensus.cc:2781] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:15:44.870836 12950 raft_consensus.cc:686] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [term 1 LEADER]: Becoming Leader. State: Replica: 1c19057b4f19476c89b2fdacd864e4cf, State: Running, Role: LEADER
I20260924 00:15:44.870921 12950 consensus_queue.cc:237] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } }
I20260924 00:15:44.871034 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.862551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44085 (local address 127.1.44.131:56769)
0924 00:15:44.862840 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.862860 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.862908 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.867720 (+ 4812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.867721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.867724 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.867795 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.867796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.868683 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.868684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.869826 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.869828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.870805 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.870806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.870891 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.870896 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.870928 (+ 32us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.870964 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":55,"threads_started":1}
I20260924 00:15:44.871614 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.862701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:56769 (local address 127.1.44.130:44085)
0924 00:15:44.867587 (+ 4886us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.867589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.867597 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:15:44.867620 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.867621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.867632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.867872 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.867916 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.868582 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.869928 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.870658 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.871470 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.871479 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.871496 (+ 17us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.871540 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4866,"thread_start_us":49,"threads_started":1}
I20260924 00:15:44.871774 12438 catalog_manager.cc:5495] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c19057b4f19476c89b2fdacd864e4cf (127.1.44.129). New cstate: current_term: 1 leader_uuid: "1c19057b4f19476c89b2fdacd864e4cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c19057b4f19476c89b2fdacd864e4cf" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 45271 } health_report { overall_health: HEALTHY } } }
I20260924 00:15:44.871869 12742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca81e50d6044d7590f7cbd9e2727356" candidate_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" is_pre_election: true
I20260924 00:15:44.872023 12742 raft_consensus.cc:2454] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b45adfc8cf4f7b8ff7889d6da1b532 in term 0.
I20260924 00:15:44.872624 12810 leader_election.cc:302] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b45adfc8cf4f7b8ff7889d6da1b532, c8bbbe202bb24d7e81a90c0ce1e50d3a; no voters:
I20260924 00:15:44.872762 12949 raft_consensus.cc:2741] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260924 00:15:44.874874 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.861522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:47107 (local address 127.1.44.130:44085)
0924 00:15:44.861620 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.861622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.861686 (+ 64us) server_negotiation.cc:407] Connection header received
0924 00:15:44.861796 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.861798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.861811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.861842 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.869635 (+ 7793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.870305 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.873976 (+ 3671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.874510 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.874550 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.874556 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.874766 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.874800 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260924 00:15:44.874982 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.862786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:38377 (local address 127.1.44.129:45271)
0924 00:15:44.862842 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.862844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.870589 (+ 7745us) server_negotiation.cc:407] Connection header received
0924 00:15:44.871181 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.871182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.871236 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.871269 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:15:44.871797 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.872457 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.874125 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.874627 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.874662 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.874667 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.874877 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.874916 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 00:15:44.875315 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.862662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:45271 (local address 127.1.44.131:38377)
0924 00:15:44.870531 (+ 7869us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.870541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.870719 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.871333 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.871334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.871336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.871386 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.871388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.872803 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.872804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.874045 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.874047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.874720 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.874721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.874790 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.874793 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.875075 (+ 282us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.875111 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7855,"thread_start_us":36,"threads_started":1}
I20260924 00:15:44.875329 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.861472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44085 (local address 127.1.44.129:47107)
0924 00:15:44.861616 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.861628 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.861712 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.861917 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:15:44.861917 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:15:44.861919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:15:44.867470 (+ 5551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.867473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.872486 (+ 5013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.872487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.873886 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:15:44.873889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.874626 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.874626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:15:44.874688 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.874691 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:15:44.875184 (+ 493us) client_negotiation.cc:240] Negotiation successful
0924 00:15:44.875237 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260924 00:15:44.875696 12742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca81e50d6044d7590f7cbd9e2727356" candidate_uuid: "1c19057b4f19476c89b2fdacd864e4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" is_pre_election: true
I20260924 00:15:44.875777 12742 raft_consensus.cc:2454] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c19057b4f19476c89b2fdacd864e4cf in term 0.
I20260924 00:15:44.875732 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca81e50d6044d7590f7cbd9e2727356" candidate_uuid: "1c19057b4f19476c89b2fdacd864e4cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a"
I20260924 00:15:44.875835 12741 raft_consensus.cc:3037] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:15:44.876189 12605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca81e50d6044d7590f7cbd9e2727356" candidate_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c19057b4f19476c89b2fdacd864e4cf" is_pre_election: true
I20260924 00:15:44.876641 12741 raft_consensus.cc:2454] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c19057b4f19476c89b2fdacd864e4cf in term 1.
I20260924 00:15:44.933620 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.930325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.1.44.129:45271)
0924 00:15:44.930389 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:15:44.930391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:15:44.930449 (+ 58us) server_negotiation.cc:407] Connection header received
0924 00:15:44.930582 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:15:44.930584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:15:44.930603 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:15:44.930653 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:15:44.930856 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.931404 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.932590 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:15:44.932931 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:15:44.933006 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:15:44.933199 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:15:44.933335 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:15:44.933380 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 00:15:44.933527 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 00:15:44.933549 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:15:44.935811 12877 raft_consensus.cc:1261] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 1 FOLLOWER]: Refusing update from remote peer 1c19057b4f19476c89b2fdacd864e4cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:15:44.935815 12741 raft_consensus.cc:1261] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 1 FOLLOWER]: Refusing update from remote peer 1c19057b4f19476c89b2fdacd864e4cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:15:44.936161 12950 consensus_queue.cc:1035] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b45adfc8cf4f7b8ff7889d6da1b532" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 43277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:15:44.936322 12958 consensus_queue.cc:1035] T 3ca81e50d6044d7590f7cbd9e2727356 P 1c19057b4f19476c89b2fdacd864e4cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8bbbe202bb24d7e81a90c0ce1e50d3a" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 44085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:15:44.940840 12438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:15:44.940951 12438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:15:44.941615 12438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7faa049e97234d089a0ec8fa49ad78b9: Sending DeleteTablet for 3 replicas of tablet 3ca81e50d6044d7590f7cbd9e2727356
I20260924 00:15:44.941920 12585 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca81e50d6044d7590f7cbd9e2727356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:15:44 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45710
I20260924 00:15:44.941944 12704 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca81e50d6044d7590f7cbd9e2727356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:15:44 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:34112
I20260924 00:15:44.942167 12857 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca81e50d6044d7590f7cbd9e2727356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:15:44 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:60770
I20260924 00:15:44.942190 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 12520
I20260924 00:15:44.944348 12981 tablet_replica.cc:330] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: stopping tablet replica
I20260924 00:15:44.944465 12981 raft_consensus.cc:2229] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:15:44.944564 12981 raft_consensus.cc:2258] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:15:44.944919 12981 ts_tablet_manager.cc:1905] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:15:44.946662 12981 ts_tablet_manager.cc:1918] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:15:44.946732 12981 log.cc:1196] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/3ca81e50d6044d7590f7cbd9e2727356
I20260924 00:15:44.946978 12981 ts_tablet_manager.cc:1939] T 3ca81e50d6044d7590f7cbd9e2727356 P 63b45adfc8cf4f7b8ff7889d6da1b532: Deleting consensus metadata
I20260924 00:15:44.947412 12426 catalog_manager.cc:4841] TS 63b45adfc8cf4f7b8ff7889d6da1b532 (127.1.44.131:43277): tablet 3ca81e50d6044d7590f7cbd9e2727356 (table test-table [id=02a64f4ea9f4420bb02af42d73d65931]) successfully deleted
I20260924 00:15:44.948798 12980 tablet_replica.cc:330] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: stopping tablet replica
I20260924 00:15:44.948904 12980 raft_consensus.cc:2229] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:15:44.949013 12980 raft_consensus.cc:2258] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:15:44.949467 12980 ts_tablet_manager.cc:1905] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:15:44.951251 12980 ts_tablet_manager.cc:1918] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260924 00:15:44.951365 12424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:45271 (error 108)
I20260924 00:15:44.951399 12980 log.cc:1196] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/3ca81e50d6044d7590f7cbd9e2727356
I20260924 00:15:44.951762 12980 ts_tablet_manager.cc:1939] T 3ca81e50d6044d7590f7cbd9e2727356 P c8bbbe202bb24d7e81a90c0ce1e50d3a: Deleting consensus metadata
I20260924 00:15:44.951870 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 12657
I20260924 00:15:44.952714 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:15:44.952544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:45271 (local address 127.0.0.1:45732)
0924 00:15:44.952630 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.952668 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:45271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":53,"threads_started":1}
W20260924 00:15:44.952834 12424 catalog_manager.cc:4584] TS 1c19057b4f19476c89b2fdacd864e4cf (127.1.44.129:45271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ca81e50d6044d7590f7cbd9e2727356: Network error: Client connection negotiation failed: client connection to 127.1.44.129:45271: connect: Connection refused (error 111)
W20260924 00:15:44.958068 12983 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:15:44.957887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:44085 (local address 127.0.0.1:34120)
0924 00:15:44.957915 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0924 00:15:44.957925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:15:44.957993 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:15:44.958031 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.130:44085: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":15}
I20260924 00:15:44.958285 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 12794
I20260924 00:15:44.963999 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 12407
I20260924 00:15:44.973634 1202 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 00:15:45.307842 1202 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 00:15:45.307941 1202 mini_ranger.cc:66] Stopping Ranger...
I20260924 00:15:45.651891 1202 mini_ranger.cc:68] Stopped Ranger
2026-09-24 00:15:45.652 UTC [11993] LOG: received smart shutdown request
2026-09-24 00:15:45.655 UTC [11993] LOG: background worker "logical replication launcher" (PID 12002) exited with exit code 1
2026-09-24 00:15:45.655 UTC [11997] LOG: shutting down
2026-09-24 00:15:45.815 UTC [11993] LOG: database system is shut down
2026-09-24T00:15:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43829 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:15:45 dist-test-slave-cm7b krb5kdc[12992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:15:45 dist-test-slave-cm7b krb5kdc[12992](info): set up 2 sockets
Sep 24 00:15:45 dist-test-slave-cm7b krb5kdc[12992](info): commencing operation
krb5kdc: starting...
W20260924 00:15:46.951155 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:15:47 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:15:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:15:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/rangeradmin_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/rangeradmin_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/rangerlookup_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/rangerlookup_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
I20260924 00:15:47.105944 1202 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskoOH61c/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/postgres -l logfile start
2026-09-24 00:15:49.538 UTC [13022] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 00:15:49.538 UTC [13022] LOG: listening on IPv4 address "127.1.44.148", port 44169
2026-09-24 00:15:49.542 UTC [13022] LOG: listening on Unix socket "/tmp/.s.PGSQL.44169"
2026-09-24 00:15:49.562 UTC [13025] LOG: database system was shut down at 2026-09-24 00:15:48 UTC
2026-09-24 00:15:49.566 UTC [13022] LOG: database system is ready to accept connections
I20260924 00:15:50.515051 1202 mini_postgres.cc:96] Postgres bound to 44169
2026-09-24 00:15:50.521 UTC [13033] FATAL: database "slave" does not exist
127.1.44.148:44169 - accepting connections
I20260924 00:15:50.521803 1202 mini_ranger.cc:161] Starting Ranger...
I20260924 00:15:50.536291 1202 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 00:15:50.972926 1202 mini_ranger.cc:87] Created ranger Postgres database
I20260924 00:15:50.972992 1202 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-admin
2026-09-24 00:15:51,249 [I] DB FLAVOR :POSTGRES
2026-09-24 00:15:51,250 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:15:51,250 [I] Checking connection..
2026-09-24 00:15:51,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 00:15:51,476 [I] Checking connection passed.
2026-09-24 00:15:51,477 [I] --------- Verifying version history table ---------
2026-09-24 00:15:51,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:15:51,735 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:15:51,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:15:51,968 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 00:15:51,969 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 00:15:51,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 00:15:52.197 UTC [13097] WARNING: there is no transaction in progress
2026-09-24 00:15:52,203 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 00:15:52,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 00:15:52,462 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 00:15:52,462 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:15:52,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 00:15:52,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 00:15:52,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cm7b.c.gcp-upstream.internal','N') ;"
2026-09-24 00:15:53,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:15:53,414 [I] Table x_portal_user does not exist in database ranger
2026-09-24 00:15:53,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:15:53,670 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 00:15:53,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:15:53,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:15:54,128 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 00:15:54,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 00:15:55.083 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.099 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.116 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.132 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.150 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.167 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.223 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.229 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.244 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.254 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.264 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.277 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.287 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.293 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.299 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55.306 UTC [13205] WARNING: there is no transaction in progress
2026-09-24 00:15:55,916 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 00:15:55,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 00:15:56,160 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 00:15:56,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 00:15:56,399 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 00:15:56,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 00:15:56,635 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 00:15:56,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 00:15:56,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 00:15:57,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cm7b.c.gcp-upstream.internal';"
2026-09-24 00:15:57,336 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 00:15:57,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 00:15:57,593 [I] DB_PATCHES have already been applied
I20260924 00:15:57.598598 1202 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-admin:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskoOH61c/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 00:15:57.598664 1202 mini_ranger.cc:193] Using host: 127.1.44.148
I20260924 00:15:57.601624 1202 mini_ranger.cc:227] Ranger admin URL: http://127.1.44.148:37475
Sep 24, 2026 12:15:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:15:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:15:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 12:15:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 12:15:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 12:15:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 12:15:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 00:15:58.713726 1202 mini_ranger.cc:160] Time spent starting Ranger: real 8.192s user 0.002s sys 0.006s
Sep 24, 2026 12:16:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 12:16:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 12:16:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 12:16:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 12:16:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 00:16:20.559728 1202 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.44.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.44.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/rangerkms_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.44.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/rangerkms_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.44.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.44.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.44.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/HTTP_127.1.44.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 00:16:20 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208980, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 00:16:20.639592 1202 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 00:16:20.648591 1202 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 00:16:21.550197 1202 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 00:16:21.550307 1202 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms
I20260924 00:16:21.550329 1202 mini_ranger_kms.cc:227] Using postgres at 127.1.44.148:44169
2026-09-24 00:16:21,713 [I] DB FLAVOR :POSTGRES
2026-09-24 00:16:21,713 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 00:16:21,713 [I] Checking connection
2026-09-24 00:16:21,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 00:16:21,943 [I] connection success
2026-09-24 00:16:21,944 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 00:16:21,944 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 00:16:21,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 00:16:22,170 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 00:16:22,170 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 00:16:22,171 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 00:16:22,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.44.148:44169/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 00:16:22,453 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 00:16:22.921923 1202 mini_ranger_kms.cc:310] Created kms service
I20260924 00:16:23.105250 1202 mini_ranger_kms.cc:326] Created kudu user
I20260924 00:16:23.150807 1202 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 00:16:23 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208980, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:23.554666 1202 mini_ranger_kms.cc:384] Added ranger policy
I20260924 00:16:23.554780 1202 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms:/tmp/dist-test-taskoOH61c/build/release/bin/postgresql.jar:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskoOH61c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 00:16:23.554797 1202 mini_ranger_kms.cc:240] Using host: 127.1.44.148
I20260924 00:16:23.558342 1202 mini_ranger_kms.cc:276] Ranger KMS PID: 13385
I20260924 00:16:23.558435 1202 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.44.148:40779
Sep 24, 2026 12:16:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 12:16:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 12:16:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 12:16:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 12:16:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 12:16:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 00:16:25.814810 1202 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.175s user 0.005s sys 0.136s
I20260924 00:16:25.814920 1202 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 00:16:25.814965 1202 mini_ranger_kms.cc:401] 127.1.44.148:40779/kms/v1/keys
Sep 24 00:16:25 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790208985, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.44.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 00:16:26 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208985, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.44.148@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
Sep 24 00:16:26 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208985, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.44.148@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
Sep 24 00:16:28 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:16:28.954581 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44717
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44717
--ranger_config_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:29.036103 13436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:29.036298 13436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:29.036319 13436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:29.037439 13436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:29.037487 13436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:29.037503 13436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:29.037516 13436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:29.037619 13436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:29.039335 13436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.44.148:40779/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44717
--ranger_config_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:29.039568 13436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:29.039794 13436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:29.044565 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:29.044561 13441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:29.044859 13444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:29.044750 13436 server_base.cc:1034] running on GCE node
I20260924 00:16:29.045130 13436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:29.045470 13436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:29.046656 13436 hybrid_clock.cc:648] HybridClock initialized: now 1790208989046625 us; error 43 us; skew 500 ppm
Sep 24 00:16:29 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208989, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:29.049669 13436 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:29.050158 13436 webserver.cc:460] Webserver started at http://127.1.44.190:38077/ using document root <none> and password file <none>
I20260924 00:16:29.050400 13436 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:29.050465 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:29.050606 13436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:16:29 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208989, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:29.100564 13436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "2cbd4929066d4afbbb7a9285fc133523"
format_stamp: "Formatted at 2026-09-24 00:16:29 on dist-test-slave-cm7b"
server_key: "d3c4561785b8e1103368ae6df71b602e"
server_key_iv: "f7f62ce1d4300954812e0fa50ad78e54"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:29.101001 13436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "2cbd4929066d4afbbb7a9285fc133523"
format_stamp: "Formatted at 2026-09-24 00:16:29 on dist-test-slave-cm7b"
server_key: "d3c4561785b8e1103368ae6df71b602e"
server_key_iv: "f7f62ce1d4300954812e0fa50ad78e54"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:29.102767 13436 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 00:16:29.117753 13451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:29.118000 13436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:16:29.118116 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "2cbd4929066d4afbbb7a9285fc133523"
format_stamp: "Formatted at 2026-09-24 00:16:29 on dist-test-slave-cm7b"
server_key: "d3c4561785b8e1103368ae6df71b602e"
server_key_iv: "f7f62ce1d4300954812e0fa50ad78e54"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:29.118221 13436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:29.125450 13436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:29.127580 13436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:29.127733 13436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:29.132709 13436 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44717
I20260924 00:16:29.132741 13503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44717 every 8 connection(s)
I20260924 00:16:29.133098 13436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:29.133858 13504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:29.136904 13504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Bootstrap starting.
I20260924 00:16:29.137558 13504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:29.137869 13504 log.cc:824] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:29.138618 13504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: No bootstrap required, opened a new log
I20260924 00:16:29.140259 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 13436
I20260924 00:16:29.140374 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:16:29.140569 1202 external_mini_cluster.cc:1428] Setting key f9ee7c3daf92cb3a19428447dd314a04
I20260924 00:16:29.141404 13504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:29.141583 13504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:29.141614 13504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cbd4929066d4afbbb7a9285fc133523, State: Initialized, Role: FOLLOWER
I20260924 00:16:29.141788 13504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:29.141893 13504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:29.141922 13504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:29.141985 13504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:29.142668 13504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:29.142774 13504 leader_election.cc:302] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cbd4929066d4afbbb7a9285fc133523; no voters:
I20260924 00:16:29.142968 13504 leader_election.cc:288] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:16:29.143019 13509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:29.143230 13509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 LEADER]: Becoming Leader. State: Replica: 2cbd4929066d4afbbb7a9285fc133523, State: Running, Role: LEADER
I20260924 00:16:29.143289 13504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:29.143405 13509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:29.143760 13509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cbd4929066d4afbbb7a9285fc133523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } } }
I20260924 00:16:29.143882 13509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:29.144069 13509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cbd4929066d4afbbb7a9285fc133523. Latest consensus state: current_term: 1 leader_uuid: "2cbd4929066d4afbbb7a9285fc133523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } } }
I20260924 00:16:29.144215 13509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:29.144368 13518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:29.144649 13504 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 00:16:29.144924 13518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:16:29 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:29.155908 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:29.141103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.1.44.190:44717)
0924 00:16:29.141347 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:29.141355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:29.141381 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:16:29.141480 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:29.141487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:29.141543 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:29.141694 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:29.142467 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:29.143855 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:29.144999 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:29.147903 (+ 2904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:29.148417 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:29.151462 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:29.151466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:29.151467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:29.151480 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:29.153617 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:29.154041 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:29.154044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:29.154045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:29.154173 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:29.154419 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:29.154420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:29.154420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:29.154815 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:29.155433 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:29.155525 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 00:16:29.155666 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
I20260924 00:16:29.160063 13518 catalog_manager.cc:1347] Generated new cluster ID: 63a1518b61374ca496c236a0c3ee03ed
I20260924 00:16:29.160128 13518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:16:29.192066 13518 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:16:29.192929 13518 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:29.202383 13518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Generated new TSK 0
I20260924 00:16:29.202683 13518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:16:29.917773 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44717
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:30.014135 13549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:30.014360 13549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:30.014396 13549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:30.015561 13549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:30.015614 13549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:30.015655 13549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:16:30.017125 13549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.44.148:40779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:30.017375 13549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:30.017623 13549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:30.023852 13557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.023890 13554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.024127 13555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:30.024142 13549 server_base.cc:1034] running on GCE node
I20260924 00:16:30.024398 13549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:30.024646 13549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:30.025837 13549 hybrid_clock.cc:648] HybridClock initialized: now 1790208990025816 us; error 33 us; skew 500 ppm
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:30.028853 13549 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:16:30.029371 13549 webserver.cc:460] Webserver started at http://127.1.44.129:40711/ using document root <none> and password file <none>
I20260924 00:16:30.029625 13549 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:30.029690 13549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:30.029824 13549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:30.044150 13549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "fe9586fe6c6f402da7f5364b1507a9c6"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c072933b12b6d14c60e34b73405b50ec"
server_key_iv: "a217a1fda73690923bede169a6f21967"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.044651 13549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "fe9586fe6c6f402da7f5364b1507a9c6"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c072933b12b6d14c60e34b73405b50ec"
server_key_iv: "a217a1fda73690923bede169a6f21967"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.046499 13549 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:30.059619 13564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.059815 13549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:30.059978 13549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "fe9586fe6c6f402da7f5364b1507a9c6"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c072933b12b6d14c60e34b73405b50ec"
server_key_iv: "a217a1fda73690923bede169a6f21967"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.060096 13549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:30.080317 13549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:30.082818 13549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:30.083024 13549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:30.083312 13549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:30.083722 13549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:30.083820 13549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.083912 13549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:30.083986 13549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.090098 13549 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:35399
I20260924 00:16:30.090557 13549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:16:30.095521 13679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:35399 every 8 connection(s)
I20260924 00:16:30.097679 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 13549
I20260924 00:16:30.097795 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:16:30.097993 1202 external_mini_cluster.cc:1428] Setting key ea58b911389cfb664ac961596a717ac6
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:30.110397 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.096324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44717 (local address 127.1.44.129:55283)
0924 00:16:30.097431 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:30.097479 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:30.098270 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:30.098684 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:30.098689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:30.098956 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:30.099281 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.099287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.100295 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.100296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.101577 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.101580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.102159 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.102160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.102294 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:30.102925 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:30.102935 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:30.105869 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:30.108631 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:30.108638 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:30.108640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:30.108976 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:30.109199 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:30.109199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:30.109200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:30.109338 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:30.109652 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:30.109659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:30.109836 (+ 177us) client_negotiation.cc:769] Sending connection context
0924 00:16:30.109957 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 00:16:30.110126 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"thread_start_us":86,"threads_started":1}
I20260924 00:16:30.111022 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.096619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:55283 (local address 127.1.44.190:44717)
0924 00:16:30.098455 (+ 1836us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:30.098458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:30.098465 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:16:30.098488 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:30.098489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:30.098500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:30.098560 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:30.099387 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.100196 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.101672 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.102010 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.102081 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:30.105983 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:30.105985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:30.105985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:30.105994 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:30.108510 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:30.109064 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:30.109065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:30.109065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:30.109131 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:30.109407 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:30.109408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:30.109409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:30.109561 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:30.109624 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:30.110805 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0924 00:16:30.110894 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1807,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:16:30.112542 13680 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44717
I20260924 00:16:30.112668 13680 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:30.112862 13680 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:30.113464 13468 ts_manager.cc:194] Registered new tserver with Master: fe9586fe6c6f402da7f5364b1507a9c6 (127.1.44.129:35399)
I20260924 00:16:30.114369 13468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:55283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:16:30.134543 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44717
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:30.253777 13686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:30.254140 13686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:30.254225 13686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:30.256011 13686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:30.256124 13686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:30.256206 13686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:16:30.258436 13686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.44.148:40779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:30.258905 13686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:30.259366 13686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:30.266124 13694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.266161 13693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.266361 13696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:30.267562 13686 server_base.cc:1034] running on GCE node
I20260924 00:16:30.267764 13686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:30.268040 13686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:30.269213 13686 hybrid_clock.cc:648] HybridClock initialized: now 1790208990269189 us; error 34 us; skew 500 ppm
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:30.276755 13686 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:16:30.277238 13686 webserver.cc:460] Webserver started at http://127.1.44.130:46787/ using document root <none> and password file <none>
I20260924 00:16:30.277503 13686 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:30.277550 13686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:30.277859 13686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:30.293228 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "39e1b185354148c09a29741f62f29a7e"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "69bc7c465ff2e98afaf447cd5afafb9c"
server_key_iv: "12cd2fce1cb7cdfdca5a13f765ef68d5"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.293694 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "39e1b185354148c09a29741f62f29a7e"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "69bc7c465ff2e98afaf447cd5afafb9c"
server_key_iv: "12cd2fce1cb7cdfdca5a13f765ef68d5"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.295491 13686 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260924 00:16:30.315351 13703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.315625 13686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 00:16:30.315716 13686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "39e1b185354148c09a29741f62f29a7e"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "69bc7c465ff2e98afaf447cd5afafb9c"
server_key_iv: "12cd2fce1cb7cdfdca5a13f765ef68d5"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.315801 13686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:30.334647 13686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:30.337065 13686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:30.337265 13686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:30.337509 13686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:30.337910 13686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:30.338001 13686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.338075 13686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:30.338135 13686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.345157 13686 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:42201
I20260924 00:16:30.345676 13686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:16:30.347854 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 13686
I20260924 00:16:30.347951 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:16:30.348133 1202 external_mini_cluster.cc:1428] Setting key 4396566c75d8c3a0d0de6de770d0d1b6
I20260924 00:16:30.350442 13816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:42201 every 8 connection(s)
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
I20260924 00:16:30.368629 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.350319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:37343 (local address 127.1.44.190:44717)
0924 00:16:30.350383 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:30.350385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:30.351458 (+ 1073us) server_negotiation.cc:407] Connection header received
0924 00:16:30.352156 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:30.352158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:30.352171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:30.352201 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:30.353036 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.353841 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.357771 (+ 3930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.358109 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.358151 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:30.365107 (+ 6956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:30.365109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:30.365109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:30.365117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:30.366804 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:30.367333 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:30.367334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:30.367334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:30.367391 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:30.367651 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:30.367652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:30.367652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:30.367806 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:30.367857 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:30.368429 (+ 572us) server_negotiation.cc:299] Negotiation successful
0924 00:16:30.368525 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 00:16:30.369371 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.350026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44717 (local address 127.1.44.130:37343)
0924 00:16:30.351330 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:30.351379 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:30.352035 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:30.352399 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:30.352403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:30.352641 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:30.352926 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.352932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.353964 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.353965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.357677 (+ 3712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.357679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.358249 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.358249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.358386 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:30.363320 (+ 4934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:30.363331 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:30.364985 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:30.366909 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:30.366916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:30.366918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:30.367244 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:30.367457 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:30.367457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:30.367458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:30.367583 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:30.368102 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:30.368108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:30.368274 (+ 166us) client_negotiation.cc:769] Sending connection context
0924 00:16:30.368960 (+ 686us) client_negotiation.cc:240] Negotiation successful
0924 00:16:30.369134 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1255,"thread_start_us":90,"threads_started":1}
I20260924 00:16:30.370143 13817 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44717
I20260924 00:16:30.370234 13817 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:30.370417 13817 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:30.370791 13468 ts_manager.cc:194] Registered new tserver with Master: 39e1b185354148c09a29741f62f29a7e (127.1.44.130:42201)
I20260924 00:16:30.371393 13468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:37343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:16:30.391743 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44717
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:30.489173 13823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:30.489403 13823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:30.489451 13823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:30.490592 13823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:30.490656 13823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:30.490701 13823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:16:30.492197 13823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.44.148:40779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:44717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:30.492465 13823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:30.492703 13823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:30.497800 13829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.497809 13831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.498092 13828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:30.498273 13823 server_base.cc:1034] running on GCE node
I20260924 00:16:30.498468 13823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:30.498734 13823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:30.499876 13823 hybrid_clock.cc:648] HybridClock initialized: now 1790208990499857 us; error 36 us; skew 500 ppm
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:30.502352 13823 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:16:30.502810 13823 webserver.cc:460] Webserver started at http://127.1.44.131:37835/ using document root <none> and password file <none>
I20260924 00:16:30.503072 13823 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:30.503118 13823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:30.503258 13823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:30.516666 13823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "f603f53082de4291918c45e8a45cd000"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c51db7de76928c05b13774546e66dd62"
server_key_iv: "056058aabed544fba941be2dda13554c"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.517166 13823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "f603f53082de4291918c45e8a45cd000"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c51db7de76928c05b13774546e66dd62"
server_key_iv: "056058aabed544fba941be2dda13554c"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.523072 13823 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260924 00:16:30.543355 13838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.543632 13823 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260924 00:16:30.543730 13823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "f603f53082de4291918c45e8a45cd000"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c51db7de76928c05b13774546e66dd62"
server_key_iv: "056058aabed544fba941be2dda13554c"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.543817 13823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:30.568384 13823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:30.573307 13823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:30.573483 13823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:30.573740 13823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:30.574128 13823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:30.574182 13823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.574249 13823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:30.574280 13823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.579744 13823 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:39129
I20260924 00:16:30.580278 13823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:16:30.581193 13951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:39129 every 8 connection(s)
I20260924 00:16:30.582604 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 13823
I20260924 00:16:30.582702 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:16:30.582877 1202 external_mini_cluster.cc:1428] Setting key ef379df45cb8a62f9b1d5e7e444cf748
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:30.590369 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.581965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:47605 (local address 127.1.44.190:44717)
0924 00:16:30.582015 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:30.582018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:30.582128 (+ 110us) server_negotiation.cc:407] Connection header received
0924 00:16:30.582826 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:30.582827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:30.582841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:30.582873 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:30.583729 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.584260 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.585310 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.585520 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.585604 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:30.587647 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:30.587649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:30.587649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:30.587656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:30.589023 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:30.589435 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:30.589436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:30.589437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:30.589493 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:30.589745 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:30.589746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:30.589746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:30.589855 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:30.589925 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:30.590200 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 00:16:30.590280 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:16:30.590565 13954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.581822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44717 (local address 127.1.44.131:47605)
0924 00:16:30.581975 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:30.582027 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:30.582689 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:30.583086 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:30.583090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:30.583370 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:30.583640 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.583645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.584367 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.584368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.585220 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.585222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.585625 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.585626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.585753 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:30.586263 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:30.586271 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:30.587551 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:30.589121 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:30.589128 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:30.589130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:30.589348 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:30.589565 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:30.589566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:30.589566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:30.589676 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:30.589917 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:30.589923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:30.590066 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 00:16:30.590167 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:16:30.590310 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260924 00:16:30.591058 13952 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44717
I20260924 00:16:30.591164 13952 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:30.591360 13952 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:30.591753 13468 ts_manager.cc:194] Registered new tserver with Master: f603f53082de4291918c45e8a45cd000 (127.1.44.131:39129)
I20260924 00:16:30.592233 13468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:47605
I20260924 00:16:30.595163 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:16:30.595822 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 13549
I20260924 00:16:30.600835 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 13686
I20260924 00:16:30.613279 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 13823
I20260924 00:16:30.619511 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 13436
I20260924 00:16:30.624271 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44717
--webserver_interface=127.1.44.190
--webserver_port=38077
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44717
--ranger_config_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:30.706207 13955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:30.706413 13955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:30.706435 13955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:30.707633 13955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:30.707674 13955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:30.707688 13955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:30.707701 13955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:30.707713 13955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:30.709293 13955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44717
--ranger_config_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=38077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:30.709546 13955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:30.709766 13955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:30.714354 13960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.714365 13963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.714437 13961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:30.714645 13955 server_base.cc:1034] running on GCE node
I20260924 00:16:30.714865 13955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:30.715155 13955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:30.716318 13955 hybrid_clock.cc:648] HybridClock initialized: now 1790208990716304 us; error 29 us; skew 500 ppm
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:30.718663 13955 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:30.719148 13955 webserver.cc:460] Webserver started at http://127.1.44.190:38077/ using document root <none> and password file <none>
I20260924 00:16:30.719394 13955 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:30.719444 13955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:30.764868 13955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:30.765750 13971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.765957 13955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:30.766057 13955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "2cbd4929066d4afbbb7a9285fc133523"
format_stamp: "Formatted at 2026-09-24 00:16:29 on dist-test-slave-cm7b"
server_key: "d3c4561785b8e1103368ae6df71b602e"
server_key_iv: "f7f62ce1d4300954812e0fa50ad78e54"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.766491 13955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:30.779284 13955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:30.781337 13955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:30.781492 13955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:30.786326 13955 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44717
I20260924 00:16:30.786350 14023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44717 every 8 connection(s)
I20260924 00:16:30.786831 13955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:30.788141 14024 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:16:30.788779 14024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Bootstrap starting.
I20260924 00:16:30.790091 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 13955
I20260924 00:16:30.790395 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:35399
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:44717
--webserver_port=40711
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:30.797305 14024 log.cc:824] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:30.799023 14024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:16:30.799371 14024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Bootstrap complete.
I20260924 00:16:30.801716 14024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:30.801976 14024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cbd4929066d4afbbb7a9285fc133523, State: Initialized, Role: FOLLOWER
I20260924 00:16:30.802189 14024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:30.802287 14024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:30.802352 14024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:30.802407 14024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:16:30.803476 14024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:30.803593 14024 leader_election.cc:302] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cbd4929066d4afbbb7a9285fc133523; no voters:
I20260924 00:16:30.803802 14024 leader_election.cc:288] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:16:30.803835 14028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:16:30.804033 14028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [term 2 LEADER]: Becoming Leader. State: Replica: 2cbd4929066d4afbbb7a9285fc133523, State: Running, Role: LEADER
I20260924 00:16:30.804149 14024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:30.804169 14028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } }
I20260924 00:16:30.804576 14030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2cbd4929066d4afbbb7a9285fc133523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } } }
I20260924 00:16:30.804768 14030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:30.804718 14028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cbd4929066d4afbbb7a9285fc133523. Latest consensus state: current_term: 2 leader_uuid: "2cbd4929066d4afbbb7a9285fc133523" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cbd4929066d4afbbb7a9285fc133523" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44717 } } }
I20260924 00:16:30.805016 14028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:30.805173 14033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:30.805727 14033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:30.805919 14033 catalog_manager.cc:1259] Loaded cluster ID: 63a1518b61374ca496c236a0c3ee03ed
I20260924 00:16:30.805999 14033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:16:30.806136 14024 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 00:16:30.809861 14033 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:30.810292 14033 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Loaded TSK: 0
I20260924 00:16:30.810612 14033 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:16:30.907069 14026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:30.907454 14026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:30.907543 14026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:30.909289 14026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:30.909401 14026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:30.909487 14026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:16:30.911633 14026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:35399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=40711
--tserver_master_addrs=127.1.44.190:44717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:30.912102 14026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:30.912438 14026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:30.919255 14064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:30.919307 14061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:30.919373 14026 server_base.cc:1034] running on GCE node
W20260924 00:16:30.919606 14062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:30.919919 14026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:30.920200 14026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:30.921376 14026 hybrid_clock.cc:648] HybridClock initialized: now 1790208990921353 us; error 35 us; skew 500 ppm
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:30.924674 14026 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:16:30.925155 14026 webserver.cc:460] Webserver started at http://127.1.44.129:40711/ using document root <none> and password file <none>
I20260924 00:16:30.925390 14026 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:30.925457 14026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:30.942122 14026 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:30.943216 14072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.943421 14026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:16:30.943501 14026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "fe9586fe6c6f402da7f5364b1507a9c6"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c072933b12b6d14c60e34b73405b50ec"
server_key_iv: "a217a1fda73690923bede169a6f21967"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:30.943954 14026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:30.982092 14026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:30.984373 14026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:30.984606 14026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:30.984889 14026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:30.985325 14026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:30.985422 14026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.985528 14026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:30.985607 14026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:30.992262 14026 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:35399
I20260924 00:16:30.993203 14026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:16:30.993669 14185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:35399 every 8 connection(s)
I20260924 00:16:30.996842 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14026
I20260924 00:16:30.997143 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:42201
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:44717
--webserver_port=46787
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:16:30 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208990, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:31.007812 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.994445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:43737 (local address 127.1.44.190:44717)
0924 00:16:30.994680 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:30.994690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:30.994905 (+ 215us) server_negotiation.cc:407] Connection header received
0924 00:16:30.995792 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:30.995801 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:30.995860 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:30.995968 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:30.996803 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.998217 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.999730 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.000369 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.000499 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.003315 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:31.003320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:31.003320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:31.003334 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:31.005562 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.006098 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.006101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.006102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.006217 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.006538 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.006539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.006540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.006869 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:31.006945 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:31.007392 (+ 447us) server_negotiation.cc:299] Negotiation successful
0924 00:16:31.007545 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260924 00:16:31.008896 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:30.994381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44717 (local address 127.1.44.129:43737)
0924 00:16:30.994765 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:30.994808 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:30.995539 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:30.996196 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:30.996199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:30.996417 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:30.996673 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.996679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:30.998352 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:30.998353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:30.999638 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:30.999641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.000612 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.000613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:31.000748 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.001367 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:31.001378 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:31.003152 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:31.005665 (+ 2513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:31.005672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:31.005674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:31.005993 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:31.006297 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:31.006298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:31.006298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:31.006435 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:31.006985 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:31.006991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:31.007160 (+ 169us) client_negotiation.cc:769] Sending connection context
0924 00:16:31.008436 (+ 1276us) client_negotiation.cc:240] Negotiation successful
0924 00:16:31.008624 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":88,"threads_started":1}
I20260924 00:16:31.009910 14186 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44717
I20260924 00:16:31.010025 14186 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.010227 14186 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
W20260924 00:16:31.011097 14186 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:44717 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 00:16:31.011175 14186 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.011272 14186 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:31.011619 14186 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.011693 14186 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
W20260924 00:16:31.011997 14186 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:16:31.103178 14190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:31.103521 14190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:31.103579 14190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:31.105283 14190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:31.105369 14190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:31.105425 14190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:16:31.107553 14190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:42201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=46787
--tserver_master_addrs=127.1.44.190:44717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:31.107923 14190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:31.108206 14190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:31.115900 14196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:31.116125 14198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:31.115908 14195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:31.116717 14190 server_base.cc:1034] running on GCE node
I20260924 00:16:31.116940 14190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:31.117208 14190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:31.118882 14190 hybrid_clock.cc:648] HybridClock initialized: now 1790208991118859 us; error 45 us; skew 500 ppm
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:31.124218 14190 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:16:31.124823 14190 webserver.cc:460] Webserver started at http://127.1.44.130:46787/ using document root <none> and password file <none>
I20260924 00:16:31.125159 14190 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:31.125303 14190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:31.139392 14190 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:31.142009 14206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:31.142244 14190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260924 00:16:31.142341 14190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "39e1b185354148c09a29741f62f29a7e"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "69bc7c465ff2e98afaf447cd5afafb9c"
server_key_iv: "12cd2fce1cb7cdfdca5a13f765ef68d5"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:31.142750 14190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:31.160271 14190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:31.162497 14190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:31.162662 14190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:31.162905 14190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:31.163280 14190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:31.163362 14190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:31.163425 14190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:31.163456 14190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:31.170552 14190 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:42201
I20260924 00:16:31.170640 14319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:42201 every 8 connection(s)
I20260924 00:16:31.170969 14190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:16:31.174304 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14190
I20260924 00:16:31.174595 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.44.148:40779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:39129
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:44717
--webserver_port=37835
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:31.181151 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:31.171842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:58437 (local address 127.1.44.190:44717)
0924 00:16:31.171886 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:31.171889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:31.172176 (+ 287us) server_negotiation.cc:407] Connection header received
0924 00:16:31.172710 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:31.172711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:31.172726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:31.172783 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:31.173325 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.173825 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.174882 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.175331 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.175404 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.177780 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:31.177782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:31.177782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:31.177789 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:31.179490 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.180047 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.180048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.180049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.180110 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.180379 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.180380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.180380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.180511 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:31.180571 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:31.180959 (+ 388us) server_negotiation.cc:299] Negotiation successful
0924 00:16:31.181055 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:31.181898 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:31.171869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44717 (local address 127.1.44.130:58437)
0924 00:16:31.172050 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:31.172091 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:31.172593 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:31.172860 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:31.172863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:31.173032 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:31.173223 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:31.173226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.173941 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.173943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:31.174795 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:31.174797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.175493 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.175494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:31.175650 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.176218 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:31.176229 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:31.177669 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:31.179644 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:31.179650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:31.179653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:31.179951 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:31.180182 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:31.180183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:31.180183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:31.180308 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:31.180624 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:31.180630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:31.180792 (+ 162us) client_negotiation.cc:769] Sending connection context
0924 00:16:31.181485 (+ 693us) client_negotiation.cc:240] Negotiation successful
0924 00:16:31.181663 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260924 00:16:31.182576 14320 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44717
I20260924 00:16:31.182668 14320 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.182857 14320 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
W20260924 00:16:31.183531 14320 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:44717 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 00:16:31.183611 14320 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.183676 14320 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:31.184062 14320 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.184167 14320 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
W20260924 00:16:31.184474 14320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:16:31.287052 14323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:31.287387 14323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:31.287441 14323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:31.289141 14323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:31.289198 14323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:31.289235 14323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:16:31.291354 14323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.44.148:40779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:39129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=37835
--tserver_master_addrs=127.1.44.190:44717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:31.291747 14323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:31.292022 14323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:31.298612 14331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:31.298971 14328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:31.299355 14329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:31.300079 14323 server_base.cc:1034] running on GCE node
I20260924 00:16:31.300313 14323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:31.300594 14323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:31.301771 14323 hybrid_clock.cc:648] HybridClock initialized: now 1790208991301756 us; error 27 us; skew 500 ppm
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:31.305090 14323 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:16:31.305840 14323 webserver.cc:460] Webserver started at http://127.1.44.131:37835/ using document root <none> and password file <none>
I20260924 00:16:31.306152 14323 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:31.306296 14323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:31.325757 14323 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:31.326414 14340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:31.326613 14323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:31.326702 14323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "f603f53082de4291918c45e8a45cd000"
format_stamp: "Formatted at 2026-09-24 00:16:30 on dist-test-slave-cm7b"
server_key: "c51db7de76928c05b13774546e66dd62"
server_key_iv: "056058aabed544fba941be2dda13554c"
server_key_version: "kuduclusterkey@0"
I20260924 00:16:31.327080 14323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:31.344734 14323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:31.347448 14323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:31.347692 14323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:31.347966 14323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:31.348455 14323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:31.348515 14323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:31.348548 14323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:31.348575 14323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:31.356118 14323 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:39129
I20260924 00:16:31.356153 14453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:39129 every 8 connection(s)
I20260924 00:16:31.356621 14323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:16:31.363229 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14323
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:31.368494 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:31.357428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:44717 (local address 127.1.44.131:56613)
0924 00:16:31.357576 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:31.357611 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:31.358123 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:31.358423 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:31.358428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:31.358653 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:31.358916 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:31.358922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.359861 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.359862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:31.361146 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:31.361149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.361662 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.361664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:31.361808 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.362407 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:31.362417 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:31.364748 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:31.366722 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:31.366729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:31.366731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:31.367054 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:31.367318 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:31.367319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:31.367319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:31.367460 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:31.367787 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:31.367794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:31.367959 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 00:16:31.368080 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 00:16:31.368243 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260924 00:16:31.369056 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:31.357407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:56613 (local address 127.1.44.190:44717)
0924 00:16:31.357470 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:31.357472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:31.357708 (+ 236us) server_negotiation.cc:407] Connection header received
0924 00:16:31.358260 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:31.358261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:31.358276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:31.358351 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:31.359034 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.359760 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.361244 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.361543 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.361624 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.364872 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:31.364874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:31.364875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:31.364883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:31.366616 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.367151 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.367152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.367152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.367250 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.367554 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.367555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.367556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.367691 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:31.367759 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:31.368881 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0924 00:16:31.368960 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 00:16:31.369633 14454 heartbeater.cc:347] Connected to a master server at 127.1.44.190:44717
I20260924 00:16:31.369726 14454 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.369906 14454 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
W20260924 00:16:31.370596 14454 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:44717 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 00:16:31.370671 14454 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.370731 14454 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:31.371085 14454 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:31.371155 14454 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
W20260924 00:16:31.371557 14454 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:16:31.372968 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:31.363657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.1.44.190:44717)
0924 00:16:31.364930 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:31.364932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:31.365070 (+ 138us) server_negotiation.cc:407] Connection header received
0924 00:16:31.365178 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:31.365179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:31.365193 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:31.365237 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:31.366080 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.366860 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.368025 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:31.368315 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:31.368356 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:31.369903 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:31.369904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:31.369905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:31.369912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:31.371691 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.372021 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.372022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.372023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.372086 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:31.372312 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:31.372313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:31.372313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:31.372456 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:31.372628 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:31.372698 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 00:16:31.372786 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1241,"thread_start_us":53,"threads_started":1}
Sep 24 00:16:31 dist-test-slave-cm7b krb5kdc[12992](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:32.012194 14186 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:32.012367 14186 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:32.012740 13988 ts_manager.cc:194] Registered new tserver with Master: fe9586fe6c6f402da7f5364b1507a9c6 (127.1.44.129:35399)
I20260924 00:16:32.013762 13988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:43737
I20260924 00:16:32.184669 14320 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:32.184857 14320 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:32.185173 13988 ts_manager.cc:194] Registered new tserver with Master: 39e1b185354148c09a29741f62f29a7e (127.1.44.130:42201)
I20260924 00:16:32.185698 13988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:58437
Sep 24 00:16:32 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
I20260924 00:16:32.371737 14454 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:32.371904 14454 heartbeater.cc:511] Master 127.1.44.190:44717 requested a full tablet report, sending...
I20260924 00:16:32.372244 13988 ts_manager.cc:194] Registered new tserver with Master: f603f53082de4291918c45e8a45cd000 (127.1.44.131:39129)
I20260924 00:16:32.372756 13988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:56613
I20260924 00:16:32.379997 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:16:32.394891 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.382137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.1.44.190:44717)
0924 00:16:32.387315 (+ 5178us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.387317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.387327 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:16:32.387353 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.387354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.387368 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.387449 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:32.388137 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.388883 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.389962 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.390253 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.390317 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.391911 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:32.391913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:32.391913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:32.391920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:32.393671 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:32.394083 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:32.394084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:32.394085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:32.394143 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:32.394347 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:32.394348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:32.394348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:32.394470 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:32.394561 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.394706 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.394789 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5151,"thread_start_us":73,"threads_started":1}
I20260924 00:16:32.397135 13988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:16:32.399567 13988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:16:32.408427 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.403854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:35399 (local address 127.0.0.1:58736)
0924 00:16:32.403955 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.403963 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:32.403996 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:32.404458 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:32.404461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:32.404464 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:32.404546 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.404549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.405707 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.405708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.407083 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.407087 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.408091 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.408092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.408182 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.408199 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 00:16:32.408286 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 00:16:32.408341 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":67,"threads_started":1}
I20260924 00:16:32.408425 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.404115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.1.44.129:35399)
0924 00:16:32.404254 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.404259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.404268 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:16:32.404303 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.404305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.404320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.404436 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:32.404626 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.405621 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.407220 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.407995 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.408072 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.408130 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.408308 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.408348 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260924 00:16:32.409410 14120 tablet_service.cc:1467] Processing CreateTablet for tablet c0ff14f0b1694287a3f6a2aec56e9c4c (DEFAULT_TABLE table=test-table [id=b1fb9d98e9d0400d8c29aa623f4a5c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:32.409778 14120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0ff14f0b1694287a3f6a2aec56e9c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:32.413810 14487 tablet_bootstrap.cc:492] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: Bootstrap starting.
I20260924 00:16:32.414515 14487 tablet_bootstrap.cc:654] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:32.414875 14487 log.cc:824] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:32.415729 14487 tablet_bootstrap.cc:492] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: No bootstrap required, opened a new log
I20260924 00:16:32.415756 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.403639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:39129 (local address 127.0.0.1:54944)
0924 00:16:32.403764 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.403774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:32.403825 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:32.411566 (+ 7741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:32.411567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:32.411570 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:32.411619 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.411621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.413329 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.413330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.414466 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.414470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.415493 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.415494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.415577 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.415583 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 00:16:32.415615 (+ 32us) client_negotiation.cc:240] Negotiation successful
0924 00:16:32.415658 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260924 00:16:32.415787 14487 ts_tablet_manager.cc:1397] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:16:32.416369 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.403738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54944 (local address 127.1.44.131:39129)
0924 00:16:32.411356 (+ 7618us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.411361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.411374 (+ 13us) server_negotiation.cc:407] Connection header received
0924 00:16:32.411412 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.411414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.411432 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.411706 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:32.411801 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.413213 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.414592 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.415405 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.416156 (+ 751us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.416224 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.416250 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.416289 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7571,"thread_start_us":60,"threads_started":1}
I20260924 00:16:32.417037 14388 tablet_service.cc:1467] Processing CreateTablet for tablet c0ff14f0b1694287a3f6a2aec56e9c4c (DEFAULT_TABLE table=test-table [id=b1fb9d98e9d0400d8c29aa623f4a5c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:32.417376 14388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0ff14f0b1694287a3f6a2aec56e9c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:32.417958 14487 raft_consensus.cc:348] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.418123 14487 raft_consensus.cc:374] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:32.418154 14487 raft_consensus.cc:729] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe9586fe6c6f402da7f5364b1507a9c6, State: Initialized, Role: FOLLOWER
I20260924 00:16:32.418262 14487 consensus_queue.cc:260] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.418503 14487 ts_tablet_manager.cc:1428] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:32.419124 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.403375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:42201 (local address 127.0.0.1:33718)
0924 00:16:32.403520 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.403559 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:32.403625 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:32.415484 (+ 11859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:32.415485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:32.415488 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:32.415525 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.415526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.416804 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.416806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.417892 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.417895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.418886 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.418887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.418951 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.418955 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:16:32.418988 (+ 33us) client_negotiation.cc:240] Negotiation successful
0924 00:16:32.419040 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260924 00:16:32.419282 14186 heartbeater.cc:503] Master 127.1.44.190:44717 was elected leader, sending a full tablet report...
I20260924 00:16:32.419719 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.403474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.1.44.130:42201)
0924 00:16:32.415286 (+ 11812us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.415291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.415303 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:16:32.415339 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.415341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.415358 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.415604 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:32.415693 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.416701 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.418002 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.418779 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.419497 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.419569 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.419598 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.419641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11773,"thread_start_us":58,"threads_started":1}
I20260924 00:16:32.420508 14254 tablet_service.cc:1467] Processing CreateTablet for tablet c0ff14f0b1694287a3f6a2aec56e9c4c (DEFAULT_TABLE table=test-table [id=b1fb9d98e9d0400d8c29aa623f4a5c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:32.420843 14254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0ff14f0b1694287a3f6a2aec56e9c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:32.421576 14490 tablet_bootstrap.cc:492] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Bootstrap starting.
I20260924 00:16:32.422626 14490 tablet_bootstrap.cc:654] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:32.422986 14490 log.cc:824] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:32.423789 14490 tablet_bootstrap.cc:492] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: No bootstrap required, opened a new log
I20260924 00:16:32.423852 14490 ts_tablet_manager.cc:1397] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:16:32.426162 14490 raft_consensus.cc:348] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.426276 14490 raft_consensus.cc:374] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:32.426296 14490 raft_consensus.cc:729] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f603f53082de4291918c45e8a45cd000, State: Initialized, Role: FOLLOWER
I20260924 00:16:32.426414 14490 consensus_queue.cc:260] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.426756 14490 ts_tablet_manager.cc:1428] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:32.426761 14454 heartbeater.cc:503] Master 127.1.44.190:44717 was elected leader, sending a full tablet report...
I20260924 00:16:32.427362 14492 tablet_bootstrap.cc:492] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: Bootstrap starting.
I20260924 00:16:32.428011 14492 tablet_bootstrap.cc:654] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:32.428296 14492 log.cc:824] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:32.429121 14492 tablet_bootstrap.cc:492] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: No bootstrap required, opened a new log
I20260924 00:16:32.429198 14492 ts_tablet_manager.cc:1397] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:32.431303 14492 raft_consensus.cc:348] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.431443 14492 raft_consensus.cc:374] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:32.431504 14492 raft_consensus.cc:729] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1b185354148c09a29741f62f29a7e, State: Initialized, Role: FOLLOWER
I20260924 00:16:32.431646 14492 consensus_queue.cc:260] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.431922 14492 ts_tablet_manager.cc:1428] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:32.431931 14320 heartbeater.cc:503] Master 127.1.44.190:44717 was elected leader, sending a full tablet report...
W20260924 00:16:32.494853 14187 tablet.cc:2369] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 00:16:32 dist-test-slave-cm7b krb5kdc[12992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208991, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for HTTP/127.1.44.148@KRBTEST.COM
W20260924 00:16:32.607730 14455 tablet.cc:2369] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:16:32.673537 14321 tablet.cc:2369] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:16:32.714558 14495 raft_consensus.cc:482] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:32.714710 14495 raft_consensus.cc:504] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.715142 14495 leader_election.cc:288] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f603f53082de4291918c45e8a45cd000 (127.1.44.131:39129), fe9586fe6c6f402da7f5364b1507a9c6 (127.1.44.129:35399)
I20260924 00:16:32.719293 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.715568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:55705 (local address 127.1.44.131:39129)
0924 00:16:32.715611 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.715613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.715787 (+ 174us) server_negotiation.cc:407] Connection header received
0924 00:16:32.715912 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.715913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.715926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.715957 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:32.716195 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.716921 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.718350 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.718870 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.718970 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.718983 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.719150 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.719213 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:16:32.719303 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.715355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:39129 (local address 127.1.44.130:55705)
0924 00:16:32.715716 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.715727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:32.715816 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:32.716030 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:32.716031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:32.716033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:32.716113 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.716116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.717024 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.717025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.718248 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.718251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.718989 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.718990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.719057 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.719065 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:16:32.719142 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 00:16:32.719184 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":10,"thread_start_us":104,"threads_started":1}
I20260924 00:16:32.719751 14408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ff14f0b1694287a3f6a2aec56e9c4c" candidate_uuid: "39e1b185354148c09a29741f62f29a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f603f53082de4291918c45e8a45cd000" is_pre_election: true
I20260924 00:16:32.719938 14408 raft_consensus.cc:2454] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e1b185354148c09a29741f62f29a7e in term 0.
I20260924 00:16:32.720160 14208 leader_election.cc:302] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1b185354148c09a29741f62f29a7e, f603f53082de4291918c45e8a45cd000; no voters:
I20260924 00:16:32.720311 14495 raft_consensus.cc:2781] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:16:32.720362 14495 raft_consensus.cc:482] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:16:32.720392 14495 raft_consensus.cc:3037] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:32.721174 14495 raft_consensus.cc:504] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.721319 14495 leader_election.cc:288] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [CANDIDATE]: Term 1 election: Requested vote from peers f603f53082de4291918c45e8a45cd000 (127.1.44.131:39129), fe9586fe6c6f402da7f5364b1507a9c6 (127.1.44.129:35399)
I20260924 00:16:32.721457 14408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ff14f0b1694287a3f6a2aec56e9c4c" candidate_uuid: "39e1b185354148c09a29741f62f29a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f603f53082de4291918c45e8a45cd000"
I20260924 00:16:32.721529 14408 raft_consensus.cc:3037] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:32.722174 14408 raft_consensus.cc:2454] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e1b185354148c09a29741f62f29a7e in term 1.
I20260924 00:16:32.722362 14208 leader_election.cc:302] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1b185354148c09a29741f62f29a7e, f603f53082de4291918c45e8a45cd000; no voters:
I20260924 00:16:32.722470 14495 raft_consensus.cc:2781] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:32.722635 14495 raft_consensus.cc:686] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 1 LEADER]: Becoming Leader. State: Replica: 39e1b185354148c09a29741f62f29a7e, State: Running, Role: LEADER
I20260924 00:16:32.722728 14495 consensus_queue.cc:237] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } }
I20260924 00:16:32.723572 13988 catalog_manager.cc:5495] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 39e1b185354148c09a29741f62f29a7e (127.1.44.130). New cstate: current_term: 1 leader_uuid: "39e1b185354148c09a29741f62f29a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1b185354148c09a29741f62f29a7e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 } health_report { overall_health: UNKNOWN } } }
I20260924 00:16:32.728402 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.715388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:35399 (local address 127.1.44.130:59795)
0924 00:16:32.715925 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.715936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:32.716039 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:32.716235 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:32.716236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:32.716237 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:32.716313 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.716315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.717231 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.717232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.727261 (+ 10029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:32.727264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.728088 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.728089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:32.728192 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.728197 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:16:32.728262 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 00:16:32.728319 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":88,"threads_started":1}
I20260924 00:16:32.728576 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.715461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:59795 (local address 127.1.44.129:35399)
0924 00:16:32.715517 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.715519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.716009 (+ 490us) server_negotiation.cc:407] Connection header received
0924 00:16:32.716126 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.716127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.716139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.716168 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:32.716392 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.717129 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.727356 (+ 10227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.727985 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.728078 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.728090 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.728456 (+ 366us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.728517 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:16:32.728942 14139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ff14f0b1694287a3f6a2aec56e9c4c" candidate_uuid: "39e1b185354148c09a29741f62f29a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe9586fe6c6f402da7f5364b1507a9c6"
I20260924 00:16:32.729367 14140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ff14f0b1694287a3f6a2aec56e9c4c" candidate_uuid: "39e1b185354148c09a29741f62f29a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" is_pre_election: true
I20260924 00:16:32.729513 14139 raft_consensus.cc:3037] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:32.730482 14139 raft_consensus.cc:2454] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e1b185354148c09a29741f62f29a7e in term 1.
I20260924 00:16:32.761375 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.758889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.1.44.130:42201)
0924 00:16:32.758952 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:32.758955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:32.758964 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:16:32.759048 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:32.759049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:32.759072 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:32.759150 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:16:32.759328 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.759816 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.760592 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:32.760803 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:32.760875 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:32.761046 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:16:32.761157 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:16:32.761248 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:32.761272 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 00:16:32.761291 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 00:16:32.763139 14139 raft_consensus.cc:1261] T c0ff14f0b1694287a3f6a2aec56e9c4c P fe9586fe6c6f402da7f5364b1507a9c6 [term 1 FOLLOWER]: Refusing update from remote peer 39e1b185354148c09a29741f62f29a7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:32.763167 14408 raft_consensus.cc:1261] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 1 FOLLOWER]: Refusing update from remote peer 39e1b185354148c09a29741f62f29a7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:32.763505 14502 consensus_queue.cc:1035] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f603f53082de4291918c45e8a45cd000" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:32.763640 14495 consensus_queue.cc:1035] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe9586fe6c6f402da7f5364b1507a9c6" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:32.765937 14510 mvcc.cc:205] Tried to move back new op lower bound from 7332696034355388416 to 7332696034192826368. Current Snapshot: MvccSnapshot[applied={T|T < 7332696034355388416}]
I20260924 00:16:32.766851 14512 mvcc.cc:205] Tried to move back new op lower bound from 7332696034355388416 to 7332696034192826368. Current Snapshot: MvccSnapshot[applied={T|T < 7332696034355388416}]
I20260924 00:16:32.767005 14511 mvcc.cc:205] Tried to move back new op lower bound from 7332696034355388416 to 7332696034192826368. Current Snapshot: MvccSnapshot[applied={T|T < 7332696034355388416}]
I20260924 00:16:32.768792 13987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:32.768905 13987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:32.769582 13987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2cbd4929066d4afbbb7a9285fc133523: Sending DeleteTablet for 3 replicas of tablet c0ff14f0b1694287a3f6a2aec56e9c4c
I20260924 00:16:32.769912 14388 tablet_service.cc:1514] Processing DeleteTablet for tablet c0ff14f0b1694287a3f6a2aec56e9c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:32 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:54944
I20260924 00:16:32.769938 14254 tablet_service.cc:1514] Processing DeleteTablet for tablet c0ff14f0b1694287a3f6a2aec56e9c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:32 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33718
I20260924 00:16:32.770143 14520 tablet_replica.cc:330] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: stopping tablet replica
I20260924 00:16:32.770188 14120 tablet_service.cc:1514] Processing DeleteTablet for tablet c0ff14f0b1694287a3f6a2aec56e9c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:32 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:58736
I20260924 00:16:32.770256 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14026
I20260924 00:16:32.770277 14520 raft_consensus.cc:2229] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:32.772292 14520 raft_consensus.cc:2258] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:32.772830 14520 ts_tablet_manager.cc:1905] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:32.774951 14520 ts_tablet_manager.cc:1918] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:32.775079 14520 log.cc:1196] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/c0ff14f0b1694287a3f6a2aec56e9c4c
I20260924 00:16:32.775432 14520 ts_tablet_manager.cc:1939] T c0ff14f0b1694287a3f6a2aec56e9c4c P f603f53082de4291918c45e8a45cd000: Deleting consensus metadata
I20260924 00:16:32.775861 13973 catalog_manager.cc:4841] TS f603f53082de4291918c45e8a45cd000 (127.1.44.131:39129): tablet c0ff14f0b1694287a3f6a2aec56e9c4c (table test-table [id=b1fb9d98e9d0400d8c29aa623f4a5c3c]) successfully deleted
W20260924 00:16:32.776793 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:35399 (error 108)
I20260924 00:16:32.777024 14519 tablet_replica.cc:330] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e: stopping tablet replica
I20260924 00:16:32.777122 14519 raft_consensus.cc:2229] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:16:32.777233 14519 raft_consensus.cc:2258] T c0ff14f0b1694287a3f6a2aec56e9c4c P 39e1b185354148c09a29741f62f29a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:32.777410 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14190
I20260924 00:16:32.778426 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.777547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:35399 (local address 127.0.0.1:58748)
0924 00:16:32.778345 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.778393 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:35399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":774}
W20260924 00:16:32.778558 13975 catalog_manager.cc:4584] TS fe9586fe6c6f402da7f5364b1507a9c6 (127.1.44.129:35399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0ff14f0b1694287a3f6a2aec56e9c4c: Network error: Client connection negotiation failed: client connection to 127.1.44.129:35399: connect: Connection refused (error 111)
I20260924 00:16:32.782907 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14323
I20260924 00:16:32.782924 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:32.782786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:42201 (local address 127.0.0.1:33742)
0924 00:16:32.782851 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:32.782884 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.130:42201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 00:16:32.787744 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 13955
I20260924 00:16:32.794147 1202 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 00:16:33.138461 1202 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 00:16:33.138594 1202 mini_ranger.cc:66] Stopping Ranger...
I20260924 00:16:33.482992 1202 mini_ranger.cc:68] Stopped Ranger
2026-09-24 00:16:33.483 UTC [13022] LOG: received smart shutdown request
2026-09-24 00:16:33.487 UTC [13022] LOG: background worker "logical replication launcher" (PID 13031) exited with exit code 1
2026-09-24 00:16:33.487 UTC [13026] LOG: shutting down
2026-09-24 00:16:33.658 UTC [13022] LOG: database system is shut down
2026-09-24T00:16:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47846 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:16:33 dist-test-slave-cm7b krb5kdc[14531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:16:33 dist-test-slave-cm7b krb5kdc[14531](info): set up 2 sockets
Sep 24 00:16:33 dist-test-slave-cm7b krb5kdc[14531](info): commencing operation
krb5kdc: starting...
W20260924 00:16:34.803272 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:16:34 dist-test-slave-cm7b krb5kdc[14531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:16:34.915902 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46447
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:46447,127.1.44.189:46875,127.1.44.188:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:34.995702 14544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:34.995942 14544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:34.995977 14544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:34.997125 14544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:34.997192 14544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:34.997217 14544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:34.997246 14544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:34.997275 14544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:34.998749 14544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:46447,127.1.44.189:46875,127.1.44.188:38065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:34.999010 14544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:34.999253 14544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:35.003852 14550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:35.003861 14552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:35.003862 14549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:35.004177 14544 server_base.cc:1034] running on GCE node
I20260924 00:16:35.004428 14544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:35.004714 14544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:35.005894 14544 hybrid_clock.cc:648] HybridClock initialized: now 1790208995005882 us; error 32 us; skew 500 ppm
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:35.008359 14544 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:35.008781 14544 webserver.cc:460] Webserver started at http://127.1.44.190:44597/ using document root <none> and password file <none>
I20260924 00:16:35.008996 14544 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:35.009097 14544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:35.009233 14544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:35.010341 14544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "1611f2c0b4ad43c497e548bc1fd87ac7"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "6f5f225f2d8bb270a797cf486294ab44"
server_key_iv: "d151c24fb36665e81c110485a015725e"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.010695 14544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "1611f2c0b4ad43c497e548bc1fd87ac7"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "6f5f225f2d8bb270a797cf486294ab44"
server_key_iv: "d151c24fb36665e81c110485a015725e"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.012035 14544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:16:35.012750 14559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:35.012907 14544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:35.013007 14544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "1611f2c0b4ad43c497e548bc1fd87ac7"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "6f5f225f2d8bb270a797cf486294ab44"
server_key_iv: "d151c24fb36665e81c110485a015725e"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.013083 14544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:35.025163 14544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:35.027099 14544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:35.027297 14544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:35.031066 14544 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:46447
I20260924 00:16:35.031106 14611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:46447 every 8 connection(s)
I20260924 00:16:35.031512 14544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:35.032382 14612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:35.033927 14612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:35.040488 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14544
I20260924 00:16:35.040611 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:16:35.040812 1202 external_mini_cluster.cc:1428] Setting key 4575087507a1985a8dbde56248be816e
I20260924 00:16:35.043417 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.034656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.1.44.190:46447)
0924 00:16:35.034841 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.034846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.034897 (+ 51us) server_negotiation.cc:407] Connection header received
0924 00:16:35.035450 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.035452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.035468 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.035567 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.036073 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.036890 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.037729 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.038038 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.038115 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.040107 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.040114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.040114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.040120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.041773 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.042195 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.042198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.042198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.042281 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.042491 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.042492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.042492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.042718 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.042803 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.043016 (+ 213us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.043126 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260924 00:16:35.043437 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.034618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46447 (local address 127.0.0.1:53134)
0924 00:16:35.034777 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.034821 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.035320 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.035591 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.035594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.035838 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.035993 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.035997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.036968 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.036969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.037656 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.037658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.038109 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.038110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.038168 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.038650 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.038658 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.040010 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.041879 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.041885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.041887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.042096 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.042347 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.042348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.042348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.042428 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.042793 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.042798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.042867 (+ 69us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.042988 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.043242 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":57,"thread_start_us":74,"threads_started":1}
I20260924 00:16:35.044442 14612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.044970 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.044821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:43970)
0924 00:16:35.044861 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.044942 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260924 00:16:35.045105 14562 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
I20260924 00:16:35.046051 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.045951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:43980)
0924 00:16:35.045984 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.046009 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 00:16:35.046144 14612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:46875: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.44.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab.
Entry for principal kudu/127.1.44.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab.
I20260924 00:16:35.065722 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:46875
--webserver_interface=127.1.44.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab
--principal=kudu/127.1.44.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:46447,127.1.44.189:46875,127.1.44.188:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:35.095366 14612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } attempt: 1
I20260924 00:16:35.095919 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.095750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:43992)
0924 00:16:35.095820 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.095877 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 00:16:35.096323 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.096224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:44006)
0924 00:16:35.096271 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.096301 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 00:16:35.096439 14612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:46875: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
W20260924 00:16:35.146170 14619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:35.146385 14619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:35.146407 14619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:35.147568 14619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:35.147611 14619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:35.147625 14619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:35.147639 14619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:35.147650 14619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:35.149107 14619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:46447,127.1.44.189:46875,127.1.44.188:38065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab
--principal=kudu/127.1.44.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.44.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:35.149341 14619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:35.149562 14619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:35.154603 14624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:35.154763 14619 server_base.cc:1034] running on GCE node
W20260924 00:16:35.154665 14625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:35.154610 14627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:35.155105 14619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:35.155433 14619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:35.156620 14619 hybrid_clock.cc:648] HybridClock initialized: now 1790208995156594 us; error 35 us; skew 500 ppm
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:35.159085 14619 init.cc:377] Logged in from keytab as kudu/127.1.44.189@KRBTEST.COM (short username kudu)
I20260924 00:16:35.159539 14619 webserver.cc:460] Webserver started at http://127.1.44.189:44919/ using document root <none> and password file <none>
I20260924 00:16:35.159760 14619 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:35.159823 14619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:35.159979 14619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:35.161065 14619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data/instance:
uuid: "3a9398cd92e34c8e82033898b0b053f1"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "2086ac0bdedcde778ff12ae9680b0d93"
server_key_iv: "d1787adeb6b31ea122ad7275be69075f"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.161423 14619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal/instance:
uuid: "3a9398cd92e34c8e82033898b0b053f1"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "2086ac0bdedcde778ff12ae9680b0d93"
server_key_iv: "d1787adeb6b31ea122ad7275be69075f"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.162711 14619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:16:35.163616 14634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:35.163813 14619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:35.163932 14619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal
uuid: "3a9398cd92e34c8e82033898b0b053f1"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "2086ac0bdedcde778ff12ae9680b0d93"
server_key_iv: "d1787adeb6b31ea122ad7275be69075f"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.164012 14619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:35.164595 14612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } attempt: 2
I20260924 00:16:35.164980 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.164835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:44010)
0924 00:16:35.164897 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.164944 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 00:16:35.165328 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.165241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:44022)
0924 00:16:35.165281 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.165307 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 00:16:35.165434 14612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:46875: Network error: Client connection negotiation failed: client connection to 127.1.44.189:46875: connect: Connection refused (error 111)
I20260924 00:16:35.188083 14619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:35.189929 14619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:35.190038 14619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:35.194092 14619 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.189:46875
I20260924 00:16:35.194154 14686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.189:46875 every 8 connection(s)
I20260924 00:16:35.194460 14619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
I20260924 00:16:35.195374 14687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:35.196938 14687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.200474 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14619
I20260924 00:16:35.200616 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-1/wal/instance
I20260924 00:16:35.200822 1202 external_mini_cluster.cc:1428] Setting key 0aac8621f4f6f45da5db00c3422127b9
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:35.206035 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.197602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46447 (local address 127.0.0.1:53136)
0924 00:16:35.197766 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.197801 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.198245 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.198450 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.198453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.198700 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.198888 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.198892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.199569 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.199570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.200465 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.200468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.200985 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.200986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.201096 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.201606 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.201614 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.202924 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.204752 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.204758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.204759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.204970 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.205193 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.205194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.205194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.205287 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.205574 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.205578 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.205685 (+ 107us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.205765 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.205879 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260924 00:16:35.206051 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.197669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.1.44.190:46447)
0924 00:16:35.197763 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.197765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.197887 (+ 122us) server_negotiation.cc:407] Connection header received
0924 00:16:35.198327 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.198328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.198342 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.198391 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.198958 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.199492 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.200576 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.200842 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.200920 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.203011 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.203012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.203013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.203020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.204674 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.205050 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.205052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.205052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.205119 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.205347 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.205348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.205348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.205506 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.205583 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.205813 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.205906 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260924 00:16:35.206871 14687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.44.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.188@KRBTEST.COM" created.
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:16:35.214939 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.207121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:44032)
0924 00:16:35.207166 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.207176 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.207238 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.207528 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.207529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.207673 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.207740 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.207742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.208562 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.208563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.209250 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.209252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.209715 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.209716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.209776 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.210152 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.210156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.211924 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.213970 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.213971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.213978 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.214138 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.214347 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.214348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.214348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.214397 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.214721 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.214721 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.214757 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.214811 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.214876 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 00:16:35.215086 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.207235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.1.44.189:46875)
0924 00:16:35.207384 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.207388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.207403 (+ 15us) server_negotiation.cc:407] Connection header received
0924 00:16:35.207419 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.207420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.207430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.207485 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.207820 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.208470 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.209316 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.209630 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.209702 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.212026 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.212034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.212034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.212041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.213884 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.214215 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.214218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.214218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.214283 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.214457 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.214458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.214459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.214650 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.214742 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.214849 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.214965 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260924 00:16:35.215720 14687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.216065 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.215964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36166)
0924 00:16:35.216007 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.216048 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 00:16:35.216190 14635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
I20260924 00:16:35.216609 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.216509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36172)
0924 00:16:35.216559 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.216584 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 00:16:35.216710 14687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38065: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab.
Entry for principal kudu/127.1.44.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab.
I20260924 00:16:35.226770 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:38065
--webserver_interface=127.1.44.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab
--principal=kudu/127.1.44.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:46447,127.1.44.189:46875,127.1.44.188:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:35.256575 14612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } attempt: 3
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:16:35.265357 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.256951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:44042)
0924 00:16:35.257017 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.257032 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.257099 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.257257 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.257258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.257439 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.257517 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.257519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.258570 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.258572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.259403 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.259406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.259982 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.259984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.260071 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.260568 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.260574 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.262176 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.264220 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.264221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.264222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.264437 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.264670 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.264671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.264671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.264737 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.265023 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.265024 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.265076 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.265158 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.265253 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 00:16:35.265734 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.257076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.1.44.189:46875)
0924 00:16:35.257124 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.257126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.257138 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:16:35.257167 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.257174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.257186 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.257253 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.257595 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.258465 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.259513 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.259896 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.259983 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.262284 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.262286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.262286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.262295 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.264128 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.264519 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.264521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.264521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.264596 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.264804 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.264805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.264806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.264952 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.265028 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.265205 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.265297 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 00:16:35.265938 14687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } attempt: 1
I20260924 00:16:35.266268 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.266175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36180)
0924 00:16:35.266217 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.266243 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 00:16:35.266276 14612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.266579 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.266492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36190)
0924 00:16:35.266533 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.266554 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 00:16:35.266579 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.266485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36198)
0924 00:16:35.266521 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.266557 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260924 00:16:35.266805 14687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38065: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
I20260924 00:16:35.266912 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.266831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36200)
0924 00:16:35.266863 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.266888 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260924 00:16:35.267012 14612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38065: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
I20260924 00:16:35.298182 14612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } attempt: 1
I20260924 00:16:35.298694 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.298527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36206)
0924 00:16:35.298599 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.298650 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260924 00:16:35.299060 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.298968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36214)
0924 00:16:35.299010 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.299039 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 00:16:35.299156 14612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38065: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
W20260924 00:16:35.319242 14694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:35.319444 14694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:35.319466 14694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:35.320593 14694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:35.320634 14694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:35.320648 14694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:35.320662 14694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:35.320674 14694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:35.322113 14694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:46447,127.1.44.189:46875,127.1.44.188:38065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab
--principal=kudu/127.1.44.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.44.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:35.322350 14694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:35.322580 14694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:35.327239 14702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:35.327322 14694 server_base.cc:1034] running on GCE node
W20260924 00:16:35.327253 14700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:35.327234 14699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:35.327710 14694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:35.327986 14694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:35.329152 14694 hybrid_clock.cc:648] HybridClock initialized: now 1790208995329143 us; error 33 us; skew 500 ppm
I20260924 00:16:35.334944 14687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } attempt: 2
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:35.335338 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.335184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36216)
0924 00:16:35.335265 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.335303 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260924 00:16:35.335608 14694 init.cc:377] Logged in from keytab as kudu/127.1.44.188@KRBTEST.COM (short username kudu)
I20260924 00:16:35.335675 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.335587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36230)
0924 00:16:35.335627 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.335654 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 00:16:35.335781 14687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38065: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38065: connect: Connection refused (error 111)
I20260924 00:16:35.336089 14694 webserver.cc:460] Webserver started at http://127.1.44.188:41601/ using document root <none> and password file <none>
I20260924 00:16:35.336306 14694 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:35.336350 14694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:35.336490 14694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:35.337587 14694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data/instance:
uuid: "e00bebbe8b7b41d3b3de99d691581edb"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "6426372fad70c3ceb7a983d01eabb91d"
server_key_iv: "4f969a4ab03b6ab86d92d6004cd6493b"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.337946 14694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal/instance:
uuid: "e00bebbe8b7b41d3b3de99d691581edb"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "6426372fad70c3ceb7a983d01eabb91d"
server_key_iv: "4f969a4ab03b6ab86d92d6004cd6493b"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.339265 14694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:16:35.340008 14709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:35.340169 14694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:35.340260 14694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal
uuid: "e00bebbe8b7b41d3b3de99d691581edb"
format_stamp: "Formatted at 2026-09-24 00:16:35 on dist-test-slave-cm7b"
server_key: "6426372fad70c3ceb7a983d01eabb91d"
server_key_iv: "4f969a4ab03b6ab86d92d6004cd6493b"
server_key_version: "encryptionkey@0"
I20260924 00:16:35.340313 14694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:35.353080 14694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:35.355288 14694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:35.355397 14694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:35.358990 14694 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.188:38065
I20260924 00:16:35.359038 14761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.188:38065 every 8 connection(s)
I20260924 00:16:35.359401 14694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
I20260924 00:16:35.360234 14762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:35.361747 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.362632 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14694
I20260924 00:16:35.362740 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790208846277299-1202-0/minicluster-data/master-2/wal/instance
I20260924 00:16:35.362921 1202 external_mini_cluster.cc:1428] Setting key 4e0c1d05875ae9e49d83a9fa34819337
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:16:35.371313 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.362389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.1.44.190:46447)
0924 00:16:35.362436 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.362438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.362638 (+ 200us) server_negotiation.cc:407] Connection header received
0924 00:16:35.363177 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.363179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.363218 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.363280 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.364054 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.364525 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.365600 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.365804 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.365886 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.368325 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.368327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.368327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.368333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.369901 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.370350 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.370351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.370351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.370420 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.370656 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.370657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.370658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.370772 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.370840 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.371118 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.371224 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:35.371594 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.362352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:46447 (local address 127.0.0.1:53138)
0924 00:16:35.362506 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.362545 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.363080 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.363406 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.363419 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.363743 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.363965 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.363969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.364603 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.364604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.365508 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.365511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.365894 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.365895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.366030 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.366562 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.366573 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.368232 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.369998 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.370004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.370006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.370278 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.370482 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.370483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.370483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.370592 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.370842 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.370848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.370981 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.371082 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.371248 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":163,"thread_start_us":89,"threads_started":1}
I20260924 00:16:35.371603 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.363320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.1.44.188:38065)
0924 00:16:35.363498 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.363502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.363658 (+ 156us) server_negotiation.cc:407] Connection header received
0924 00:16:35.363682 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.363684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.363709 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.363783 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.364340 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.365130 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.365840 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.366145 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.366231 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.368790 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.368797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.368797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.368804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.370250 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.370598 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.370601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.370601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.370657 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.370856 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.370857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.370857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.371008 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.371183 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.371271 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.371352 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
I20260924 00:16:35.372148 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.374315 14612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } attempt: 2
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:16:35.379423 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.372448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.1.44.189:46875)
0924 00:16:35.372493 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.372495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.372528 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:16:35.372600 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.372601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.372610 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.372659 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.372899 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.373471 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.374159 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.374462 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.374527 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.376493 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.376494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.376495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.376502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.378072 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.378484 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.378486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.378486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.378544 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.378780 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.378781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.378782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.378931 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.378982 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.379229 (+ 247us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.379307 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:16:35.379715 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.372399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:46875 (local address 127.0.0.1:44052)
0924 00:16:35.372479 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.372490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.372540 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.372661 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.372662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.372785 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.372828 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.372829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.373540 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.373541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.374087 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.374089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.374525 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.374525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.374599 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.374987 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.374991 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.376394 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.378162 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.378163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.378163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.378386 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.378620 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.378620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.378621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.378692 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.379034 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.379035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.379082 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.379504 (+ 422us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.379599 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260924 00:16:35.380044 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:16:35.381645 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.372474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.1.44.190:46447)
0924 00:16:35.372508 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.372510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.372518 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:16:35.372577 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.372578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.372588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.372635 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.373158 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.373631 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.374317 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.374595 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.374665 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.377973 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.377975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.377975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.377982 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.379638 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.380282 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.380283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.380283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.380344 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.380570 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.380571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.380571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.380699 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.381408 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.381462 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.381545 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:16:35.382166 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.374525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36258)
0924 00:16:35.374597 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.374607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.374638 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.374862 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.374863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.375036 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.375076 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.375077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.375847 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.375849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.376590 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.376593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.377072 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.377073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.377136 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.377657 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.377663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.378970 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.381179 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.381180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.381181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.381344 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.381548 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.381549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.381549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.381605 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.381878 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.381879 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.381931 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.381981 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.382073 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 00:16:35.382560 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.374709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.1.44.188:38065)
0924 00:16:35.374748 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.374750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.374756 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:16:35.374774 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.374774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.374783 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.374828 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.375138 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.375764 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.376677 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.376990 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.377060 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.379062 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.379063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.379064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.379070 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.381006 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.381427 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.381428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.381429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.381480 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.381670 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.381671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.381671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.381802 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.382353 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.382392 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.382472 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:16:35.384199 14612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: Bootstrap starting.
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:16:35.384866 14612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:35.385186 14612 log.cc:824] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:35.386087 14612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: No bootstrap required, opened a new log
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:16:35.387902 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.380304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36262)
0924 00:16:35.380382 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.380391 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.380436 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.380639 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.380640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.380899 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.380934 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.380937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.382194 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.382195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.382833 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.382835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.383290 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.383291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.383354 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.383727 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.383731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.385006 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.386814 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.386815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.386816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.386987 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.387213 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.387214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.387214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.387266 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.387566 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.387567 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.387605 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.387694 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.387783 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260924 00:16:35.388085 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.380401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.1.44.188:38065)
0924 00:16:35.380539 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.380541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.380547 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:16:35.380562 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.380563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.380571 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.380634 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.381029 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.382090 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.382921 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.383189 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.383290 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.385065 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.385066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.385067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.385073 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.386734 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.387066 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.387067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.387068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.387118 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.387330 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.387331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.387331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.387482 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.387568 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.387716 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.387831 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260924 00:16:35.388643 14612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.388779 14612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:35.388809 14612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1611f2c0b4ad43c497e548bc1fd87ac7, State: Initialized, Role: FOLLOWER
I20260924 00:16:35.388926 14612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.389256 14769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.389380 14769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:16:35.389508 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb: Bootstrap starting.
I20260924 00:16:35.390249 14612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:35.390285 14762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:35.390587 14762 log.cc:824] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:35.391459 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb: No bootstrap required, opened a new log
W20260924 00:16:35.392196 14781 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:16:35.392293 14781 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:16:35.393492 14762 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.393642 14762 raft_consensus.cc:374] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:35.393685 14762 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00bebbe8b7b41d3b3de99d691581edb, State: Initialized, Role: FOLLOWER
I20260924 00:16:35.393813 14762 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.394030 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.382189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.1.44.189:46875)
0924 00:16:35.382224 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.382225 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.382232 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:16:35.382484 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.382485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.382494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.382551 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.383391 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.383997 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.385030 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.385309 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.385399 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.387920 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.387921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.387921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.387928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.392190 (+ 4262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.392625 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.392627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.392627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.392685 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.392894 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.392895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.392895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.393026 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.393557 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.393822 (+ 265us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.393918 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:35.394120 14762 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:35.393036 14544 master_runner.cc:186] Error getting master registration for 127.1.44.189:46875: OK, instance_id { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" instance_seqno: 1790208995192942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:16:35.396052 14544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:16:35.395671 14783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.396359 14783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: This master's current role is: FOLLOWER
W20260924 00:16:35.395967 14794 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260924 00:16:35.397145 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:16:35.398603 14694 master_runner.cc:186] Error getting master registration for 127.1.44.189:46875: OK, instance_id { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" instance_seqno: 1790208995192942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:16:35.400974 14694 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 00:16:35.397035 14794 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:16:35.406545 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.398470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.1.44.190:46447)
0924 00:16:35.398507 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.398509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.398760 (+ 251us) server_negotiation.cc:407] Connection header received
0924 00:16:35.398850 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.398851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.398860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.398889 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.399652 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.400339 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.401331 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.401627 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.401677 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.403669 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.403670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.403670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.403677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.405280 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.405648 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.405649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.405650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.405703 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.405916 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.405917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.405917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.406053 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.406149 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.406366 (+ 217us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.406439 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:16:35.409600 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.401301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.1.44.188:38065)
0924 00:16:35.401346 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.401348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.402094 (+ 746us) server_negotiation.cc:407] Connection header received
0924 00:16:35.402154 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.402154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.402167 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.402224 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.402725 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.403573 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.404576 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.404931 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.404999 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.406374 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.406375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.406375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.406381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.408420 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.408777 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.408778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.408779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.408832 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.409007 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.409008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.409008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.409141 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.409235 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.409425 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.409501 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:16:35.413138 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.401787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.1.44.189:46875)
0924 00:16:35.401826 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.401828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.406820 (+ 4992us) server_negotiation.cc:407] Connection header received
0924 00:16:35.406921 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.406922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.406934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.406964 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.407542 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.408258 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.409261 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.409545 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.409598 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.410812 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.410813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.410814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.410820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.412063 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.412397 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.412399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.412399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.412451 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.412641 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.412642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.412643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.412749 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.412859 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.412959 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.413031 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:16:35.426942 14687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } attempt: 3
Sep 24 00:16:35 dist-test-slave-cm7b krb5kdc[14531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208995, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:16:35.433805 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.427246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38065 (local address 127.0.0.1:36282)
0924 00:16:35.427300 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:35.427312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:35.427359 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:35.427515 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:35.427515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:35.427672 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:35.427711 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.427712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.428322 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.428323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.428808 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:35.428810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.429156 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.429158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:35.429213 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.429642 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:35.429647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:35.430857 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:35.432803 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.432804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.432804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.432981 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.433177 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:35.433178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:35.433179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:35.433239 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:35.433484 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:35.433485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:35.433530 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 00:16:35.433611 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 00:16:35.433709 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 00:16:35.433805 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:35.427279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.1.44.188:38065)
0924 00:16:35.427316 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:35.427319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:35.427345 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:16:35.427430 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:35.427431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:35.427441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:35.427511 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:35.427782 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.428246 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.428899 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:35.429089 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:35.429161 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:35.430940 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:35.430941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:35.430941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:35.430949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:35.432704 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.433061 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.433062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.433062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.433116 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:35.433299 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:35.433300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:35.433300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:35.433408 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:35.433491 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:35.433635 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 00:16:35.433709 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:35.435587 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: Bootstrap starting.
I20260924 00:16:35.436245 14687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:35.436547 14687 log.cc:824] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:35.437806 14687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: No bootstrap required, opened a new log
I20260924 00:16:35.439167 14687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.439344 14687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:35.439389 14687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a9398cd92e34c8e82033898b0b053f1, State: Initialized, Role: FOLLOWER
I20260924 00:16:35.439519 14687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.439790 14804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.439846 14687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:35.439919 14804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 00:16:35.440622 14640 tablet.cc:2369] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:16:35.441730 14815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:16:35.441849 14815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:16:35.443619 14619 master_runner.cc:263] No leader master found from master 3a9398cd92e34c8e82033898b0b053f1
I20260924 00:16:35.443672 14619 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:16:35.500836 14783 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:35.500965 14783 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.501206 14783 leader_election.cc:288] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1611f2c0b4ad43c497e548bc1fd87ac7 (127.1.44.190:46447), 3a9398cd92e34c8e82033898b0b053f1 (127.1.44.189:46875)
I20260924 00:16:35.501530 14661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00bebbe8b7b41d3b3de99d691581edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9398cd92e34c8e82033898b0b053f1" is_pre_election: true
I20260924 00:16:35.501679 14661 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00bebbe8b7b41d3b3de99d691581edb in term 0.
I20260924 00:16:35.501888 14712 leader_election.cc:302] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9398cd92e34c8e82033898b0b053f1, e00bebbe8b7b41d3b3de99d691581edb; no voters:
I20260924 00:16:35.502022 14783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:16:35.502095 14783 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:16:35.502123 14783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:35.503041 14783 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.503335 14661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00bebbe8b7b41d3b3de99d691581edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a9398cd92e34c8e82033898b0b053f1"
I20260924 00:16:35.503422 14661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:35.504168 14661 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00bebbe8b7b41d3b3de99d691581edb in term 1.
I20260924 00:16:35.504336 14712 leader_election.cc:302] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a9398cd92e34c8e82033898b0b053f1, e00bebbe8b7b41d3b3de99d691581edb; no voters:
I20260924 00:16:35.504769 14586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00bebbe8b7b41d3b3de99d691581edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" is_pre_election: true
I20260924 00:16:35.504937 14586 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e00bebbe8b7b41d3b3de99d691581edb in term 0.
I20260924 00:16:35.505072 14585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e00bebbe8b7b41d3b3de99d691581edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7"
I20260924 00:16:35.505146 14585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:35.506001 14585 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e00bebbe8b7b41d3b3de99d691581edb in term 1.
I20260924 00:16:35.503170 14783 leader_election.cc:288] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [CANDIDATE]: Term 1 election: Requested vote from peers 1611f2c0b4ad43c497e548bc1fd87ac7 (127.1.44.190:46447), 3a9398cd92e34c8e82033898b0b053f1 (127.1.44.189:46875)
I20260924 00:16:35.506313 14783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:35.506366 14783 raft_consensus.cc:686] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [term 1 LEADER]: Becoming Leader. State: Replica: e00bebbe8b7b41d3b3de99d691581edb, State: Running, Role: LEADER
I20260924 00:16:35.506464 14783 consensus_queue.cc:237] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } }
I20260924 00:16:35.506843 14783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00bebbe8b7b41d3b3de99d691581edb. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.506942 14783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:35.507159 14819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:35.508486 14819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:35.510303 14661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [term 1 FOLLOWER]: Refusing update from remote peer e00bebbe8b7b41d3b3de99d691581edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:35.510596 14783 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:35.511718 14804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00bebbe8b7b41d3b3de99d691581edb. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.511823 14804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:16:35.512022 14585 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [term 1 FOLLOWER]: Refusing update from remote peer e00bebbe8b7b41d3b3de99d691581edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:35.512253 14783 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:35.513298 14769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00bebbe8b7b41d3b3de99d691581edb. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.513410 14769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:16:35.514277 14817 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.514364 14817 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:35.514678 14769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.514755 14769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:16:35.514990 14820 mvcc.cc:205] Tried to move back new op lower bound from 7332696045608325120 to 7332696045595115520. Current Snapshot: MvccSnapshot[applied={T|T < 7332696045608325120}]
I20260924 00:16:35.515502 14823 mvcc.cc:205] Tried to move back new op lower bound from 7332696045608325120 to 7332696045595115520. Current Snapshot: MvccSnapshot[applied={T|T < 7332696045608325120}]
I20260924 00:16:35.517009 14819 catalog_manager.cc:1347] Generated new cluster ID: 95b31afced65443b896541ac0a83e0d9
I20260924 00:16:35.517061 14819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:16:35.528479 14783 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.528591 14783 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:35.529021 14804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e00bebbe8b7b41d3b3de99d691581edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1611f2c0b4ad43c497e548bc1fd87ac7" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46447 } } peers { permanent_uuid: "3a9398cd92e34c8e82033898b0b053f1" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 46875 } } peers { permanent_uuid: "e00bebbe8b7b41d3b3de99d691581edb" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38065 } } }
I20260924 00:16:35.529119 14804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:16:35.529212 14821 mvcc.cc:205] Tried to move back new op lower bound from 7332696045608325120 to 7332696045595115520. Current Snapshot: MvccSnapshot[applied={T|T < 7332696045608325120}]
I20260924 00:16:35.544289 14819 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:16:35.545035 14819 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:35.556706 14819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e00bebbe8b7b41d3b3de99d691581edb: Generated new TSK 0
I20260924 00:16:35.556913 14819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:16:36.392792 14781 catalog_manager.cc:1259] Loaded cluster ID: 95b31afced65443b896541ac0a83e0d9
I20260924 00:16:36.392866 14781 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: loading cluster ID for follower catalog manager: success
I20260924 00:16:36.393712 14781 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: acquiring CA information for follower catalog manager: success
I20260924 00:16:36.394070 14781 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1611f2c0b4ad43c497e548bc1fd87ac7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:16:36.442302 14815 catalog_manager.cc:1259] Loaded cluster ID: 95b31afced65443b896541ac0a83e0d9
I20260924 00:16:36.442384 14815 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: loading cluster ID for follower catalog manager: success
I20260924 00:16:36.443393 14815 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: acquiring CA information for follower catalog manager: success
I20260924 00:16:36.443707 14815 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a9398cd92e34c8e82033898b0b053f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:16:36.618681 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14544
I20260924 00:16:36.623941 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14619
I20260924 00:16:36.628526 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14694
2026-09-24T00:16:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2856 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:16:36 dist-test-slave-cm7b krb5kdc[14834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:16:36 dist-test-slave-cm7b krb5kdc[14834](info): set up 2 sockets
Sep 24 00:16:36 dist-test-slave-cm7b krb5kdc[14834](info): commencing operation
krb5kdc: starting...
W20260924 00:16:37.651631 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:16:37 dist-test-slave-cm7b krb5kdc[14834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:16:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:16:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:16:37.769833 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44197
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:32771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:37.848591 14847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:37.848784 14847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:37.848805 14847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:37.849859 14847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:37.849890 14847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:37.849905 14847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:37.849917 14847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:37.849929 14847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:37.851369 14847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:32771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:37.851562 14847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:37.851723 14847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:16:37.856325 14847 server_base.cc:1034] running on GCE node
W20260924 00:16:37.856309 14853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:37.856312 14852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:37.856288 14855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:37.856805 14847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:37.857049 14847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:37.858217 14847 hybrid_clock.cc:648] HybridClock initialized: now 1790208997858204 us; error 40 us; skew 500 ppm
Sep 24 00:16:37 dist-test-slave-cm7b krb5kdc[14834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208997, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:37.860677 14847 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:37.861121 14847 webserver.cc:460] Webserver started at http://127.1.44.190:43863/ using document root <none> and password file <none>
I20260924 00:16:37.861338 14847 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:37.861400 14847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:37.861545 14847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:37.862556 14847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "60323f3295e6448fadadefdd9b9418a4"
format_stamp: "Formatted at 2026-09-24 00:16:37 on dist-test-slave-cm7b"
server_key: "b88b1d1db4b1aa67e9d5392f6611945d"
server_key_iv: "1bc4fff3d1526482ccaed4b71fb1915d"
server_key_version: "encryptionkey@0"
I20260924 00:16:37.862895 14847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "60323f3295e6448fadadefdd9b9418a4"
format_stamp: "Formatted at 2026-09-24 00:16:37 on dist-test-slave-cm7b"
server_key: "b88b1d1db4b1aa67e9d5392f6611945d"
server_key_iv: "1bc4fff3d1526482ccaed4b71fb1915d"
server_key_version: "encryptionkey@0"
I20260924 00:16:37.864266 14847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:16:37.864917 14862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:37.865067 14847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:16:37.865163 14847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "60323f3295e6448fadadefdd9b9418a4"
format_stamp: "Formatted at 2026-09-24 00:16:37 on dist-test-slave-cm7b"
server_key: "b88b1d1db4b1aa67e9d5392f6611945d"
server_key_iv: "1bc4fff3d1526482ccaed4b71fb1915d"
server_key_version: "encryptionkey@0"
I20260924 00:16:37.865229 14847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:37.875322 14847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:37.877447 14847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:37.877604 14847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:37.881507 14847 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44197
I20260924 00:16:37.881621 14914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44197 every 8 connection(s)
I20260924 00:16:37.881899 14847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:37.882691 14915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:37.884210 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14847
I20260924 00:16:37.884320 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:16:37.884497 1202 external_mini_cluster.cc:1428] Setting key 92a137379e9b804dc3ff13054c3bbe77
I20260924 00:16:37.885296 14915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4: Bootstrap starting.
I20260924 00:16:37.885988 14915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:37.886273 14915 log.cc:824] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:37.886906 14915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4: No bootstrap required, opened a new log
I20260924 00:16:37.888681 14915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60323f3295e6448fadadefdd9b9418a4" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44197 } }
I20260924 00:16:37.888831 14915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:37.888875 14915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60323f3295e6448fadadefdd9b9418a4, State: Initialized, Role: FOLLOWER
I20260924 00:16:37.889006 14915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60323f3295e6448fadadefdd9b9418a4" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44197 } }
I20260924 00:16:37.889097 14915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:37.889142 14915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:37.889189 14915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:37.889681 14915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60323f3295e6448fadadefdd9b9418a4" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44197 } }
I20260924 00:16:37.889793 14915 leader_election.cc:302] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60323f3295e6448fadadefdd9b9418a4; no voters:
I20260924 00:16:37.889984 14915 leader_election.cc:288] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:16:37.890007 14920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:37.890120 14920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [term 1 LEADER]: Becoming Leader. State: Replica: 60323f3295e6448fadadefdd9b9418a4, State: Running, Role: LEADER
I20260924 00:16:37.890200 14920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60323f3295e6448fadadefdd9b9418a4" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44197 } }
I20260924 00:16:37.890232 14915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:37.890514 14920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60323f3295e6448fadadefdd9b9418a4. Latest consensus state: current_term: 1 leader_uuid: "60323f3295e6448fadadefdd9b9418a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60323f3295e6448fadadefdd9b9418a4" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44197 } } }
I20260924 00:16:37.890594 14920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:37.890527 14922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60323f3295e6448fadadefdd9b9418a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60323f3295e6448fadadefdd9b9418a4" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44197 } } }
I20260924 00:16:37.890765 14922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:37.890974 14925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 00:16:37 dist-test-slave-cm7b krb5kdc[14834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:37.891556 14925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:37.893100 14925 catalog_manager.cc:1347] Generated new cluster ID: 1531433df0624c2c9226f2023d5ae205
I20260924 00:16:37.893157 14925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:16:37.898262 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:37.884955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.1.44.190:44197)
0924 00:16:37.885154 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:37.885165 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:37.885197 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:16:37.885781 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:37.885790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:37.885848 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:37.886002 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:37.886642 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:37.887659 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:37.888370 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:37.888768 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:37.888898 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:37.891582 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:37.891586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:37.891586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:37.891597 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:37.896655 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:37.897131 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:37.897134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:37.897134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:37.897228 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:37.897425 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:37.897426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:37.897426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:37.897696 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:37.897835 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:37.897949 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 00:16:37.898080 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260924 00:16:37.898962 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:16:37.906536 14925 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:16:37.907009 14925 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:37.911034 14925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60323f3295e6448fadadefdd9b9418a4: Generated new TSK 0
I20260924 00:16:37.911226 14925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:16:37.952857 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14847
2026-09-24T00:16:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1323 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:16:38 dist-test-slave-cm7b krb5kdc[14942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:16:38 dist-test-slave-cm7b krb5kdc[14942](info): set up 2 sockets
Sep 24 00:16:38 dist-test-slave-cm7b krb5kdc[14942](info): commencing operation
krb5kdc: starting...
W20260924 00:16:38.979300 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:16:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:16:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:16:39.095494 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42855
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:39.177019 14955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:39.177233 14955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:39.177254 14955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:39.178390 14955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:39.178431 14955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:39.178445 14955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:39.178479 14955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:39.178510 14955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:39.180003 14955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:42855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:39.180238 14955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:39.180451 14955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:39.185140 14961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:39.185106 14960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:39.185249 14955 server_base.cc:1034] running on GCE node
W20260924 00:16:39.185106 14963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:39.185513 14955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:39.185802 14955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:39.186955 14955 hybrid_clock.cc:648] HybridClock initialized: now 1790208999186943 us; error 38 us; skew 500 ppm
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:39.189410 14955 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:39.189847 14955 webserver.cc:460] Webserver started at http://127.1.44.190:34855/ using document root <none> and password file <none>
I20260924 00:16:39.190023 14955 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:39.190068 14955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:39.190155 14955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:39.191134 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "e2299082a9ed49a4b986eb1419947344"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c80f93bba8b0261df1cf5f9365e079c0"
server_key_iv: "212714c81c1424d12641729aabf7021b"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.191502 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "e2299082a9ed49a4b986eb1419947344"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c80f93bba8b0261df1cf5f9365e079c0"
server_key_iv: "212714c81c1424d12641729aabf7021b"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.192713 14955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:39.193354 14970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.193502 14955 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:39.193590 14955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "e2299082a9ed49a4b986eb1419947344"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c80f93bba8b0261df1cf5f9365e079c0"
server_key_iv: "212714c81c1424d12641729aabf7021b"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.193655 14955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:39.214272 14955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:39.216192 14955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:39.216300 14955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:39.220057 14955 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:42855
I20260924 00:16:39.220108 15022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:42855 every 8 connection(s)
I20260924 00:16:39.220425 14955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:39.221392 15023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:39.223745 15023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap starting.
I20260924 00:16:39.224304 15023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:39.224586 15023 log.cc:824] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:39.225241 15023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: No bootstrap required, opened a new log
I20260924 00:16:39.226758 15023 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:39.226912 15023 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:39.226958 15023 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2299082a9ed49a4b986eb1419947344, State: Initialized, Role: FOLLOWER
I20260924 00:16:39.227124 15023 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:39.227224 15023 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:39.227267 15023 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:39.227316 15023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:39.228097 15023 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:39.228210 15023 leader_election.cc:302] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2299082a9ed49a4b986eb1419947344; no voters:
I20260924 00:16:39.228374 15023 leader_election.cc:288] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:16:39.228402 15026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:39.228572 15026 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 LEADER]: Becoming Leader. State: Replica: e2299082a9ed49a4b986eb1419947344, State: Running, Role: LEADER
I20260924 00:16:39.228638 15023 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:39.228664 15026 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:39.229089 15027 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2299082a9ed49a4b986eb1419947344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } } }
I20260924 00:16:39.229101 15028 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2299082a9ed49a4b986eb1419947344. Latest consensus state: current_term: 1 leader_uuid: "e2299082a9ed49a4b986eb1419947344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } } }
I20260924 00:16:39.229218 15027 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:39.229223 15028 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:39.229755 15035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:39.230204 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 14955
I20260924 00:16:39.230261 15035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:39.230335 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:16:39.230531 1202 external_mini_cluster.cc:1428] Setting key e225b991829a0c37dbe575b94fca53ea
I20260924 00:16:39.232218 15035 catalog_manager.cc:1347] Generated new cluster ID: 9cc695db42f64f9cbb3a6d95b1455d29
I20260924 00:16:39.232270 15035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:39.240082 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.231172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.1.44.190:42855)
0924 00:16:39.231468 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.231476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.231507 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:16:39.231627 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.231634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.231686 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.231820 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:39.232491 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.233407 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.234140 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.234584 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.234700 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.236985 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:39.236989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:39.236989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:39.237000 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:39.238601 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.238948 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.238951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.238952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.239053 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.239280 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.239281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.239281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.239518 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:39.239678 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.239757 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.239895 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":105,"threads_started":1}
I20260924 00:16:39.249100 15035 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:16:39.249599 15035 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:39.258172 15035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Generated new TSK 0
I20260924 00:16:39.258378 15035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:16:39.292444 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42855
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:39.398290 15049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:39.398492 15049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:39.398515 15049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:39.399734 15049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:39.399776 15049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:39.399804 15049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:16:39.401183 15049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:39.401358 15049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:39.401527 15049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:39.406776 15057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:39.406955 15049 server_base.cc:1034] running on GCE node
W20260924 00:16:39.406996 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:39.406772 15054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:39.407372 15049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:39.407649 15049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:39.408814 15049 hybrid_clock.cc:648] HybridClock initialized: now 1790208999408781 us; error 48 us; skew 500 ppm
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:39.411486 15049 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:16:39.411880 15049 webserver.cc:460] Webserver started at http://127.1.44.129:41395/ using document root <none> and password file <none>
I20260924 00:16:39.412134 15049 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:39.412205 15049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:39.412352 15049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:39.413416 15049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "de90510f61444ecbb91f3949ac022ced"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "696862627848222aa8ca4b489808a4bf"
server_key_iv: "a3cb1ecd77867f5295969999373971f0"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.413775 15049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "de90510f61444ecbb91f3949ac022ced"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "696862627848222aa8ca4b489808a4bf"
server_key_iv: "a3cb1ecd77867f5295969999373971f0"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.415426 15049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:16:39.416193 15064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.416368 15049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:39.416458 15049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "de90510f61444ecbb91f3949ac022ced"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "696862627848222aa8ca4b489808a4bf"
server_key_iv: "a3cb1ecd77867f5295969999373971f0"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.416539 15049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:39.444221 15049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:39.446697 15049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:39.446872 15049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:39.447129 15049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:39.447470 15049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:39.447528 15049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.447572 15049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:39.447600 15049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.453670 15049 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:41629
I20260924 00:16:39.453738 15177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:41629 every 8 connection(s)
I20260924 00:16:39.454063 15049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:16:39.458104 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15049
I20260924 00:16:39.458191 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:16:39.458415 1202 external_mini_cluster.cc:1428] Setting key 434248485262080082e06162b2228e95
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:39.464061 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.454759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:52237 (local address 127.1.44.190:42855)
0924 00:16:39.454802 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.454804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.455052 (+ 248us) server_negotiation.cc:407] Connection header received
0924 00:16:39.455650 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.455652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.455666 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.455727 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:39.456745 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.457258 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.458337 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.458577 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.458672 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.461242 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:39.461244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:39.461244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:39.461251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:39.462657 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.463081 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.463082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.463083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.463141 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.463408 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.463409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.463410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.463525 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:39.463599 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.463878 (+ 279us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.463973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:39.464253 15180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.454751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:52237)
0924 00:16:39.454923 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.454964 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.455544 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.455799 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.455804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.456386 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:39.456580 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.456584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.457362 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.457363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.458245 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.458248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.458712 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.458713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.458867 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.459487 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:39.459498 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:39.461129 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:39.462764 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:39.462770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:39.462772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:39.462993 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:39.463221 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:39.463222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:39.463222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:39.463338 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:39.463592 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:39.463598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:39.463741 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.463842 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.463999 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260924 00:16:39.464699 15178 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42855
I20260924 00:16:39.464792 15178 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:39.464967 15178 heartbeater.cc:511] Master 127.1.44.190:42855 requested a full tablet report, sending...
I20260924 00:16:39.465481 14987 ts_manager.cc:194] Registered new tserver with Master: de90510f61444ecbb91f3949ac022ced (127.1.44.129:41629)
I20260924 00:16:39.466089 14987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:52237
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:16:39.482014 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42855
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:39.564265 15183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:39.564465 15183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:39.564489 15183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:39.565578 15183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:39.565615 15183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:39.565641 15183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:16:39.567111 15183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:39.567358 15183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:39.567549 15183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:39.572458 15188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:39.572506 15189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:39.572458 15191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:39.572768 15183 server_base.cc:1034] running on GCE node
I20260924 00:16:39.572933 15183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:39.573135 15183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:39.574267 15183 hybrid_clock.cc:648] HybridClock initialized: now 1790208999574256 us; error 26 us; skew 500 ppm
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:39.576821 15183 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:16:39.577296 15183 webserver.cc:460] Webserver started at http://127.1.44.130:35019/ using document root <none> and password file <none>
I20260924 00:16:39.577541 15183 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:39.577587 15183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:39.577752 15183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:39.578792 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "8a1ddd082abd48b686827116192d1876"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c2df08da05cae1bfd4f10a30f11714b4"
server_key_iv: "26f1d653ced83fd2f06a4d31288d43d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.579149 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "8a1ddd082abd48b686827116192d1876"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c2df08da05cae1bfd4f10a30f11714b4"
server_key_iv: "26f1d653ced83fd2f06a4d31288d43d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.580487 15183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:16:39.581218 15198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.581401 15183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.581473 15183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "8a1ddd082abd48b686827116192d1876"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c2df08da05cae1bfd4f10a30f11714b4"
server_key_iv: "26f1d653ced83fd2f06a4d31288d43d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.581549 15183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:39.588617 15183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:39.590392 15183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:39.590545 15183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:39.590752 15183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:39.591053 15183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:39.591107 15183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.591147 15183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:39.591177 15183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.597195 15183 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:37723
I20260924 00:16:39.597240 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:37723 every 8 connection(s)
I20260924 00:16:39.597589 15183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:39.606057 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.598258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:59791 (local address 127.1.44.190:42855)
0924 00:16:39.598331 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.598333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.598549 (+ 216us) server_negotiation.cc:407] Connection header received
0924 00:16:39.599030 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.599031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.599040 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.599093 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:39.600029 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.600517 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.601460 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.601639 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.601689 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.603793 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:39.603794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:39.603795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:39.603801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:39.604974 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.605319 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.605320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.605320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.605367 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.605567 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.605568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.605568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.605668 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:39.605716 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.605934 (+ 218us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.605993 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 00:16:39.606213 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.598246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:59791)
0924 00:16:39.598420 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.598460 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.598939 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.599149 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.599152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.599744 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:39.599926 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.599931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.600601 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.600601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.601368 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.601369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.601708 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.601709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.601804 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.602308 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:39.602315 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:39.603712 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:39.605045 (+ 1333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:39.605052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:39.605054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:39.605256 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:39.605423 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:39.605423 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:39.605424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:39.605514 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:39.605721 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:39.605725 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:39.605830 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.605909 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.606020 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":107,"threads_started":1}
I20260924 00:16:39.606494 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15183
I20260924 00:16:39.606643 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:16:39.606758 15312 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42855
I20260924 00:16:39.606863 15312 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:39.606881 1202 external_mini_cluster.cc:1428] Setting key e8f522f02fe0cb95fedb201adb3d3e9e
I20260924 00:16:39.607059 15312 heartbeater.cc:511] Master 127.1.44.190:42855 requested a full tablet report, sending...
I20260924 00:16:39.607501 14986 ts_manager.cc:194] Registered new tserver with Master: 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723)
I20260924 00:16:39.607877 14986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:59791
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:16:39.629918 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42855
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:39.712263 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:39.712478 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:39.712502 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:39.713639 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:39.713680 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:39.713706 15317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:16:39.715131 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:39.715399 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:39.715667 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:16:39.720754 15317 server_base.cc:1034] running on GCE node
W20260924 00:16:39.720727 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:39.720721 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:39.720721 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:39.721140 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:39.721374 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:39.722523 15317 hybrid_clock.cc:648] HybridClock initialized: now 1790208999722506 us; error 32 us; skew 500 ppm
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:39.724996 15317 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:16:39.725453 15317 webserver.cc:460] Webserver started at http://127.1.44.131:42329/ using document root <none> and password file <none>
I20260924 00:16:39.725680 15317 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:39.725749 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:39.725898 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:39.726933 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "260b0cfbc64c433e9c22e210c321bbdb"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "8d6823fcb910ee92be60f9060893ed79"
server_key_iv: "74ac25e97bc35d4cb13c26c7d2f728d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.727349 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "260b0cfbc64c433e9c22e210c321bbdb"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "8d6823fcb910ee92be60f9060893ed79"
server_key_iv: "74ac25e97bc35d4cb13c26c7d2f728d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.728691 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:16:39.729477 15332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.729672 15317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:16:39.729768 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "260b0cfbc64c433e9c22e210c321bbdb"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "8d6823fcb910ee92be60f9060893ed79"
server_key_iv: "74ac25e97bc35d4cb13c26c7d2f728d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:39.729820 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:39.755007 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:39.757366 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:39.757481 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:39.757673 15317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:39.758035 15317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:39.758091 15317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.758158 15317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:39.758178 15317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:39.763988 15317 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:38081
I20260924 00:16:39.764055 15445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:38081 every 8 connection(s)
I20260924 00:16:39.764360 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:16:39.764653 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15317
I20260924 00:16:39.764757 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:16:39.764935 1202 external_mini_cluster.cc:1428] Setting key a74209d6933ac4b8944ad32c22b9c753
Sep 24 00:16:39 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790208999, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:39.773720 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.765340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:46357 (local address 127.1.44.190:42855)
0924 00:16:39.765386 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.765388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.765661 (+ 273us) server_negotiation.cc:407] Connection header received
0924 00:16:39.766276 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.766276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.766285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.766330 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:39.767271 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.767786 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.768767 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.768963 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.769022 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.771534 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:39.771535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:39.771536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:39.771543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:39.772667 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.773012 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.773013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.773013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.773062 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.773237 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.773237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.773238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.773332 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:39.773379 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.773598 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.773655 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:16:39.773937 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.765352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:46357)
0924 00:16:39.765528 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.765589 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.766192 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.766381 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.766385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.766934 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:39.767134 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.767138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.767878 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.767879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.768685 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.768687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.769048 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.769048 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.769159 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.769704 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:39.769713 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:39.771453 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:39.772737 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:39.772742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:39.772744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:39.772947 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:39.773101 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:39.773102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:39.773102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:39.773189 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:39.773386 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:39.773391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:39.773497 (+ 106us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.773579 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.773690 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260924 00:16:39.774469 15446 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42855
I20260924 00:16:39.774580 15446 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:39.774748 15446 heartbeater.cc:511] Master 127.1.44.190:42855 requested a full tablet report, sending...
I20260924 00:16:39.775113 14986 ts_manager.cc:194] Registered new tserver with Master: 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
I20260924 00:16:39.775506 14986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:46357
I20260924 00:16:39.776644 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:16:39.783480 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.777877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.1.44.190:42855)
0924 00:16:39.777958 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.777960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.778032 (+ 72us) server_negotiation.cc:407] Connection header received
0924 00:16:39.778106 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.778107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.778116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.778186 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:39.778743 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.779241 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.779958 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.780151 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.780230 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.781261 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:39.781263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:39.781263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:39.781269 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:39.782321 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.782671 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.782672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.782672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.782728 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:39.782915 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:39.782916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:39.782916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:39.783059 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:39.783158 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.783294 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.783367 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 00:16:39.785168 14986 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:16:39.786717 14986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:16:39.793030 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.789145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.1.44.129:41629)
0924 00:16:39.789298 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.789303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.789410 (+ 107us) server_negotiation.cc:407] Connection header received
0924 00:16:39.789452 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.789454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.789470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.789551 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:39.789729 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.790532 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792045 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.792598 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.792756 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.792904 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.792940 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260924 00:16:39.793021 15458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.788901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41629 (local address 127.0.0.1:53694)
0924 00:16:39.789341 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.789349 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.789394 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.789566 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.789567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.789569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:39.789642 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.789644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.790655 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.790656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.791957 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.791959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.792700 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.792779 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.792792 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.792888 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.792939 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":28,"thread_start_us":88,"threads_started":1}
I20260924 00:16:39.793244 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.788888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37723 (local address 127.0.0.1:45440)
0924 00:16:39.789142 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.789181 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.789245 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.789480 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.789482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.789484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:39.789556 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.789558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.790916 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.790917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.792015 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.792017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.793005 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.793006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.793072 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.793076 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.793129 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.793178 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260924 00:16:39.793532 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.789065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.1.44.130:37723)
0924 00:16:39.789334 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.789339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.789349 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:16:39.789386 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.789388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.789402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.789477 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:39.789642 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.790819 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792098 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.792839 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792915 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.792976 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.793409 (+ 433us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.793455 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":80,"threads_started":1}
I20260924 00:16:39.793588 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.788889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:38081 (local address 127.0.0.1:45160)
0924 00:16:39.789227 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.789237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.789292 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.789889 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.789890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.789892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:39.789925 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.789927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.791110 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.791111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.792113 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.792115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.793007 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.793007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.793408 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.793415 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.793481 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.793517 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260924 00:16:39.793768 15112 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9983a4a09b4779ad11413d338c037a (DEFAULT_TABLE table=test-table [id=c0cfd712b53c48728b043668dc4330d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:39.794152 15112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9983a4a09b4779ad11413d338c037a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:39.794144 15246 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9983a4a09b4779ad11413d338c037a (DEFAULT_TABLE table=test-table [id=c0cfd712b53c48728b043668dc4330d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:39.794427 15246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9983a4a09b4779ad11413d338c037a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:39.795392 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.789055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45160 (local address 127.1.44.131:38081)
0924 00:16:39.789613 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.789617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.789626 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:16:39.789659 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.789727 (+ 68us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.789745 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.789816 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:39.789987 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.790998 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792193 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.792861 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.792918 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.792968 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.793504 (+ 536us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.793552 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":73,"threads_started":1}
I20260924 00:16:39.796012 15380 tablet_service.cc:1467] Processing CreateTablet for tablet 1a9983a4a09b4779ad11413d338c037a (DEFAULT_TABLE table=test-table [id=c0cfd712b53c48728b043668dc4330d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:39.796319 15380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a9983a4a09b4779ad11413d338c037a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:39.797736 15463 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Bootstrap starting.
I20260924 00:16:39.798416 15463 tablet_bootstrap.cc:654] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:39.799001 15463 log.cc:824] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:39.799592 15464 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Bootstrap starting.
I20260924 00:16:39.799642 15466 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Bootstrap starting.
I20260924 00:16:39.799836 15463 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: No bootstrap required, opened a new log
I20260924 00:16:39.799911 15463 ts_tablet_manager.cc:1397] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 00:16:39.800210 15464 tablet_bootstrap.cc:654] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:39.800391 15466 tablet_bootstrap.cc:654] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:39.800745 15464 log.cc:824] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:39.800951 15466 log.cc:824] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:39.801602 15464 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: No bootstrap required, opened a new log
I20260924 00:16:39.801666 15466 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: No bootstrap required, opened a new log
I20260924 00:16:39.801673 15464 ts_tablet_manager.cc:1397] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:39.801736 15466 ts_tablet_manager.cc:1397] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:16:39.802088 15463 raft_consensus.cc:348] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.802276 15463 raft_consensus.cc:374] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:39.802310 15463 raft_consensus.cc:729] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1ddd082abd48b686827116192d1876, State: Initialized, Role: FOLLOWER
I20260924 00:16:39.802450 15463 consensus_queue.cc:260] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.802729 15463 ts_tablet_manager.cc:1428] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 00:16:39.802839 15312 heartbeater.cc:503] Master 127.1.44.190:42855 was elected leader, sending a full tablet report...
I20260924 00:16:39.803670 15464 raft_consensus.cc:348] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.803838 15464 raft_consensus.cc:374] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:39.803882 15464 raft_consensus.cc:729] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de90510f61444ecbb91f3949ac022ced, State: Initialized, Role: FOLLOWER
I20260924 00:16:39.803752 15466 raft_consensus.cc:348] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.803939 15466 raft_consensus.cc:374] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:39.803984 15466 raft_consensus.cc:729] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260b0cfbc64c433e9c22e210c321bbdb, State: Initialized, Role: FOLLOWER
I20260924 00:16:39.803987 15464 consensus_queue.cc:260] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.804077 15466 consensus_queue.cc:260] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.804275 15464 ts_tablet_manager.cc:1428] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:39.804284 15178 heartbeater.cc:503] Master 127.1.44.190:42855 was elected leader, sending a full tablet report...
I20260924 00:16:39.804347 15466 ts_tablet_manager.cc:1428] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:39.804507 15446 heartbeater.cc:503] Master 127.1.44.190:42855 was elected leader, sending a full tablet report...
W20260924 00:16:39.848153 15313 tablet.cc:2369] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:16:39.900833 15470 raft_consensus.cc:482] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:39.900982 15470 raft_consensus.cc:504] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.901438 15470 leader_election.cc:288] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
I20260924 00:16:39.905485 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.901789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:41295 (local address 127.1.44.131:38081)
0924 00:16:39.901833 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.901835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.901983 (+ 148us) server_negotiation.cc:407] Connection header received
0924 00:16:39.902013 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.902014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.902029 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.902067 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:39.902236 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.902918 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.904468 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.905060 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.905146 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.905154 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.905372 (+ 218us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.905408 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:16:39.905478 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.901733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:47809 (local address 127.1.44.130:37723)
0924 00:16:39.901807 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.901809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.901820 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:16:39.901852 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.901853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.901864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.901953 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:39.902153 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.902905 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.904462 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.905060 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.905146 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.905154 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.905372 (+ 218us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.905408 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:16:39.905498 15180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.901649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37723 (local address 127.1.44.129:47809)
0924 00:16:39.901719 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.901731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.901777 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.901959 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.901960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.901962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:39.902043 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.902045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.902998 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.902999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.904343 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.904346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.905173 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.905175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.905282 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.905287 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.905357 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.905407 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":97}
I20260924 00:16:39.905519 15472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.901683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:38081 (local address 127.1.44.129:41295)
0924 00:16:39.901869 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.901883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:39.901943 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:39.902097 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:39.902098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:39.902100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:39.902161 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.902162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.902998 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.902999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.904343 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:39.904346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.905173 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.905175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:39.905282 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.905287 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:16:39.905357 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 00:16:39.905407 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":63,"thread_start_us":112,"threads_started":1}
I20260924 00:16:39.905889 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9983a4a09b4779ad11413d338c037a" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1ddd082abd48b686827116192d1876" is_pre_election: true
I20260924 00:16:39.906061 15266 raft_consensus.cc:2454] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 0.
I20260924 00:16:39.906309 15068 leader_election.cc:302] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a1ddd082abd48b686827116192d1876, de90510f61444ecbb91f3949ac022ced; no voters:
I20260924 00:16:39.906397 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9983a4a09b4779ad11413d338c037a" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260b0cfbc64c433e9c22e210c321bbdb" is_pre_election: true
I20260924 00:16:39.906453 15470 raft_consensus.cc:2781] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:16:39.906498 15470 raft_consensus.cc:482] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:16:39.906529 15470 raft_consensus.cc:3037] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:39.906541 15400 raft_consensus.cc:2454] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 0.
I20260924 00:16:39.907385 15470 raft_consensus.cc:504] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.907529 15470 leader_election.cc:288] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 election: Requested vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
I20260924 00:16:39.907747 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9983a4a09b4779ad11413d338c037a" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1ddd082abd48b686827116192d1876"
I20260924 00:16:39.907759 15400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a9983a4a09b4779ad11413d338c037a" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260b0cfbc64c433e9c22e210c321bbdb"
I20260924 00:16:39.907838 15400 raft_consensus.cc:3037] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:39.907838 15266 raft_consensus.cc:3037] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:39.908710 15400 raft_consensus.cc:2454] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 1.
I20260924 00:16:39.908710 15266 raft_consensus.cc:2454] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 1.
I20260924 00:16:39.908908 15068 leader_election.cc:302] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a1ddd082abd48b686827116192d1876, de90510f61444ecbb91f3949ac022ced; no voters:
I20260924 00:16:39.909013 15470 raft_consensus.cc:2781] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:39.909190 15470 raft_consensus.cc:686] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 LEADER]: Becoming Leader. State: Replica: de90510f61444ecbb91f3949ac022ced, State: Running, Role: LEADER
I20260924 00:16:39.909296 15470 consensus_queue.cc:237] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:39.910135 14986 catalog_manager.cc:5495] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced reported cstate change: term changed from 0 to 1, leader changed from <none> to de90510f61444ecbb91f3949ac022ced (127.1.44.129). New cstate: current_term: 1 leader_uuid: "de90510f61444ecbb91f3949ac022ced" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } health_report { overall_health: HEALTHY } } }
I20260924 00:16:39.940182 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.937728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.1.44.129:41629)
0924 00:16:39.937776 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:39.937779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:39.937790 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:16:39.937865 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:39.937866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:39.937886 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:39.937962 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:16:39.938117 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.938584 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.939381 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:39.939591 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:39.939665 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:39.939843 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:16:39.939950 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:16:39.940050 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:39.940069 (+ 19us) server_negotiation.cc:299] Negotiation successful
0924 00:16:39.940089 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:16:39.942013 15266 raft_consensus.cc:1261] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Refusing update from remote peer de90510f61444ecbb91f3949ac022ced: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:39.942013 15400 raft_consensus.cc:1261] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Refusing update from remote peer de90510f61444ecbb91f3949ac022ced: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:39.942595 15473 consensus_queue.cc:1035] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:39.942744 15470 consensus_queue.cc:1035] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [LEADER]: Connected to new peer: Peer: permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:39.945405 15478 mvcc.cc:205] Tried to move back new op lower bound from 7332696063759622144 to 7332696063629000704. Current Snapshot: MvccSnapshot[applied={T|T < 7332696063759622144}]
I20260924 00:16:39.946194 15477 mvcc.cc:205] Tried to move back new op lower bound from 7332696063759622144 to 7332696063629000704. Current Snapshot: MvccSnapshot[applied={T|T < 7332696063759622144}]
I20260924 00:16:39.947764 14986 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:39.947870 14986 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:39.948611 14986 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Sending DeleteTablet for 3 replicas of tablet 1a9983a4a09b4779ad11413d338c037a
I20260924 00:16:39.948943 15246 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a9983a4a09b4779ad11413d338c037a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:39 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45440
I20260924 00:16:39.948951 15380 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a9983a4a09b4779ad11413d338c037a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:39 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45160
I20260924 00:16:39.949177 15487 tablet_replica.cc:330] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: stopping tablet replica
I20260924 00:16:39.949258 15112 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a9983a4a09b4779ad11413d338c037a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:39 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:53694
I20260924 00:16:39.949288 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15049
I20260924 00:16:39.949298 15487 raft_consensus.cc:2229] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:39.949434 15487 raft_consensus.cc:2258] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:39.949626 15486 tablet_replica.cc:330] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: stopping tablet replica
I20260924 00:16:39.949738 15486 raft_consensus.cc:2229] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:39.949852 15486 raft_consensus.cc:2258] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:39.949990 15487 ts_tablet_manager.cc:1905] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:39.950241 15486 ts_tablet_manager.cc:1905] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:39.951637 15487 ts_tablet_manager.cc:1918] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:39.951712 15487 log.cc:1196] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/1a9983a4a09b4779ad11413d338c037a
I20260924 00:16:39.951817 15486 ts_tablet_manager.cc:1918] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:39.951881 15486 log.cc:1196] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/1a9983a4a09b4779ad11413d338c037a
I20260924 00:16:39.952008 15487 ts_tablet_manager.cc:1939] T 1a9983a4a09b4779ad11413d338c037a P 8a1ddd082abd48b686827116192d1876: Deleting consensus metadata
I20260924 00:16:39.952132 15486 ts_tablet_manager.cc:1939] T 1a9983a4a09b4779ad11413d338c037a P 260b0cfbc64c433e9c22e210c321bbdb: Deleting consensus metadata
I20260924 00:16:39.952435 14974 catalog_manager.cc:4841] TS 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): tablet 1a9983a4a09b4779ad11413d338c037a (table test-table [id=c0cfd712b53c48728b043668dc4330d0]) successfully deleted
I20260924 00:16:39.952700 14972 catalog_manager.cc:4841] TS 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): tablet 1a9983a4a09b4779ad11413d338c037a (table test-table [id=c0cfd712b53c48728b043668dc4330d0]) successfully deleted
W20260924 00:16:39.955611 14973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:41629 (error 108)
I20260924 00:16:39.955816 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15183
I20260924 00:16:39.956045 15457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:39.955937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41629 (local address 127.0.0.1:53716)
0924 00:16:39.955981 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:39.956009 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 00:16:39.956372 14973 catalog_manager.cc:4584] TS de90510f61444ecbb91f3949ac022ced (127.1.44.129:41629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a9983a4a09b4779ad11413d338c037a: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41629: connect: Connection refused (error 111)
I20260924 00:16:39.960783 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15317
I20260924 00:16:39.965170 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 14955
I20260924 00:16:39.970151 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42855
--webserver_interface=127.1.44.190
--webserver_port=34855
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:40.049183 15489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:40.049383 15489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:40.049405 15489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:40.050693 15489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:40.050740 15489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:40.050753 15489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:40.050766 15489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:40.050778 15489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:40.052325 15489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:42855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=34855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:40.052600 15489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:40.052832 15489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:40.061239 15498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.061244 15495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.061285 15496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.061496 15489 server_base.cc:1034] running on GCE node
I20260924 00:16:40.061746 15489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:40.062060 15489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:40.063272 15489 hybrid_clock.cc:648] HybridClock initialized: now 1790209000063259 us; error 25 us; skew 500 ppm
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:40.067461 15489 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:40.067878 15489 webserver.cc:460] Webserver started at http://127.1.44.190:34855/ using document root <none> and password file <none>
I20260924 00:16:40.068081 15489 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:40.068131 15489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:40.069394 15489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.069985 15505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.070137 15489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:40.070222 15489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "e2299082a9ed49a4b986eb1419947344"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c80f93bba8b0261df1cf5f9365e079c0"
server_key_iv: "212714c81c1424d12641729aabf7021b"
server_key_version: "encryptionkey@0"
I20260924 00:16:40.070601 15489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:40.094609 15489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:40.096846 15489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:40.096966 15489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:40.101235 15489 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:42855
I20260924 00:16:40.101310 15557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:42855 every 8 connection(s)
I20260924 00:16:40.101714 15489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:40.103466 15558 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:16:40.104130 15558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap starting.
I20260924 00:16:40.104723 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15489
I20260924 00:16:40.105010 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:41629
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:42855
--webserver_port=41395
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:40.112792 15558 log.cc:824] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:40.115438 15558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:16:40.115778 15558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap complete.
I20260924 00:16:40.119028 15558 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.119397 15558 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2299082a9ed49a4b986eb1419947344, State: Initialized, Role: FOLLOWER
I20260924 00:16:40.119568 15558 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.119629 15558 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:40.119678 15558 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:40.119722 15558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:16:40.121037 15558 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.121146 15558 leader_election.cc:302] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2299082a9ed49a4b986eb1419947344; no voters:
I20260924 00:16:40.121321 15558 leader_election.cc:288] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:16:40.121443 15562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:16:40.121567 15558 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:40.121603 15562 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 LEADER]: Becoming Leader. State: Replica: e2299082a9ed49a4b986eb1419947344, State: Running, Role: LEADER
I20260924 00:16:40.121701 15562 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.122051 15563 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e2299082a9ed49a4b986eb1419947344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } } }
I20260924 00:16:40.122125 15563 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:40.122443 15564 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2299082a9ed49a4b986eb1419947344. Latest consensus state: current_term: 2 leader_uuid: "e2299082a9ed49a4b986eb1419947344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } } }
I20260924 00:16:40.122557 15572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:40.122574 15564 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:40.123447 15572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:40.123545 15572 catalog_manager.cc:1259] Loaded cluster ID: 9cc695db42f64f9cbb3a6d95b1455d29
I20260924 00:16:40.123579 15572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:16:40.123811 15572 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 00:16:40.128756 15572 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 00:16:40.131430 15572 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:40.131687 15572 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Loaded TSK: 0
I20260924 00:16:40.131910 15572 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:16:40.193890 15560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:40.194128 15560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:40.194168 15560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:40.195341 15560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:40.195405 15560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:40.195449 15560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:16:40.196890 15560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:41629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=41395
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:40.197149 15560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:40.197365 15560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:40.203365 15585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.203377 15586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.203536 15588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.203649 15560 server_base.cc:1034] running on GCE node
I20260924 00:16:40.203861 15560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:40.204104 15560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:40.205271 15560 hybrid_clock.cc:648] HybridClock initialized: now 1790209000205258 us; error 30 us; skew 500 ppm
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:40.207851 15560 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:16:40.208331 15560 webserver.cc:460] Webserver started at http://127.1.44.129:41395/ using document root <none> and password file <none>
I20260924 00:16:40.208575 15560 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:40.208631 15560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:40.209962 15560 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.210659 15595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.210841 15560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.210906 15560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "de90510f61444ecbb91f3949ac022ced"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "696862627848222aa8ca4b489808a4bf"
server_key_iv: "a3cb1ecd77867f5295969999373971f0"
server_key_version: "encryptionkey@0"
I20260924 00:16:40.211321 15560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:40.229256 15560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:40.231667 15560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:40.231812 15560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:40.232079 15560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:40.232568 15602 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 00:16:40.234021 15560 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 00:16:40.234074 15560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 00:16:40.234100 15560 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 00:16:40.234743 15560 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 00:16:40.234786 15560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 00:16:40.234831 15602 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Bootstrap starting.
I20260924 00:16:40.242408 15560 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:41629
I20260924 00:16:40.242571 15709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:41629 every 8 connection(s)
I20260924 00:16:40.242936 15560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:16:40.246717 15602 log.cc:824] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:40.249225 15602 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:16:40.249532 15602 tablet_bootstrap.cc:492] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Bootstrap complete.
I20260924 00:16:40.250589 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15560
I20260924 00:16:40.250727 15602 ts_tablet_manager.cc:1397] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260924 00:16:40.250892 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:37723
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:42855
--webserver_port=35019
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:40.252487 15602 raft_consensus.cc:348] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.252823 15602 raft_consensus.cc:729] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de90510f61444ecbb91f3949ac022ced, State: Initialized, Role: FOLLOWER
I20260924 00:16:40.252976 15602 consensus_queue.cc:260] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.253273 15602 ts_tablet_manager.cc:1428] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:40.258587 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.243847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:32929 (local address 127.1.44.190:42855)
0924 00:16:40.244041 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.244050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.244094 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:16:40.244994 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.245001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.245092 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.245260 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:40.246269 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.247788 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.249286 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.249865 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.249970 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.254425 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:40.254429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:40.254429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:40.254441 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:40.256507 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.257054 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.257057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.257058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.257174 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.257501 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.257502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.257503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.257782 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:40.257867 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.258260 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.258397 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
I20260924 00:16:40.258622 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.243735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:32929)
0924 00:16:40.243923 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.243965 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.244750 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.245346 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.245352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.245767 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:40.246123 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.246131 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.247921 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.247922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.249181 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.249183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.249980 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.249981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.250127 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.252442 (+ 2315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:40.252451 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:40.254309 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:40.256601 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:40.256608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:40.256610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:40.256947 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:40.257255 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:40.257256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:40.257256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:40.257424 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:40.257865 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:40.257871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:40.258054 (+ 183us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.258184 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.258379 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260924 00:16:40.259404 15710 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42855
I20260924 00:16:40.259500 15710 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:40.259680 15710 heartbeater.cc:511] Master 127.1.44.190:42855 requested a full tablet report, sending...
I20260924 00:16:40.260457 15522 ts_manager.cc:194] Registered new tserver with Master: de90510f61444ecbb91f3949ac022ced (127.1.44.129:41629)
I20260924 00:16:40.261755 15522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:32929
I20260924 00:16:40.263010 15710 heartbeater.cc:503] Master 127.1.44.190:42855 was elected leader, sending a full tablet report...
I20260924 00:16:40.270228 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.262510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.1.44.129:41629)
0924 00:16:40.267339 (+ 4829us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.267343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.267352 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:16:40.267381 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.267382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.267394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.267487 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:40.267654 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.268370 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.269366 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.269827 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.269918 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.269974 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.270118 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.270156 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4793,"thread_start_us":69,"threads_started":1}
I20260924 00:16:40.270228 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.262224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41629 (local address 127.0.0.1:53732)
0924 00:16:40.263239 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.263279 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.263342 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.267496 (+ 4154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.267499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.267501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:40.267566 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.267568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.268463 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.268464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.269274 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.269276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.269919 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.269919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.270001 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.270013 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.270104 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.270158 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":96,"threads_started":1}
I20260924 00:16:40.270670 15644 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a9983a4a09b4779ad11413d338c037a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:39 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:53732
I20260924 00:16:40.271271 15719 tablet_replica.cc:330] stopping tablet replica
I20260924 00:16:40.271381 15719 raft_consensus.cc:2229] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:40.271451 15719 raft_consensus.cc:2258] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:40.271757 15719 ts_tablet_manager.cc:1905] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:40.274084 15719 ts_tablet_manager.cc:1918] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:40.274160 15719 log.cc:1196] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/1a9983a4a09b4779ad11413d338c037a
I20260924 00:16:40.274400 15719 ts_tablet_manager.cc:1939] T 1a9983a4a09b4779ad11413d338c037a P de90510f61444ecbb91f3949ac022ced: Deleting consensus metadata
I20260924 00:16:40.274930 15508 catalog_manager.cc:4841] TS de90510f61444ecbb91f3949ac022ced (127.1.44.129:41629): tablet 1a9983a4a09b4779ad11413d338c037a (table test-table [id=c0cfd712b53c48728b043668dc4330d0]) successfully deleted
W20260924 00:16:40.336020 15715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:40.336213 15715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:40.336236 15715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:40.337287 15715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:40.337318 15715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:40.337344 15715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:16:40.338657 15715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:37723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=35019
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:40.338821 15715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:40.338979 15715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:40.343751 15725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.343847 15715 server_base.cc:1034] running on GCE node
W20260924 00:16:40.343731 15724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.343752 15727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.344129 15715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:40.344398 15715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:40.345548 15715 hybrid_clock.cc:648] HybridClock initialized: now 1790209000345530 us; error 34 us; skew 500 ppm
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:40.350826 15715 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:16:40.351306 15715 webserver.cc:460] Webserver started at http://127.1.44.130:35019/ using document root <none> and password file <none>
I20260924 00:16:40.351495 15715 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:40.351547 15715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:40.352769 15715 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.353291 15734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.353440 15715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:40.353515 15715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "8a1ddd082abd48b686827116192d1876"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c2df08da05cae1bfd4f10a30f11714b4"
server_key_iv: "26f1d653ced83fd2f06a4d31288d43d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:40.353821 15715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:40.362574 15715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:40.364347 15715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:40.364451 15715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:40.364629 15715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:40.364910 15715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:40.364943 15715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.364984 15715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:40.365010 15715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.370633 15715 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:37723
I20260924 00:16:40.370707 15847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:37723 every 8 connection(s)
I20260924 00:16:40.371068 15715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:16:40.376617 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15715
I20260924 00:16:40.376885 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:38081
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:42855
--webserver_port=42329
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:40.380841 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.371787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:54805 (local address 127.1.44.190:42855)
0924 00:16:40.371836 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.371838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.372052 (+ 214us) server_negotiation.cc:407] Connection header received
0924 00:16:40.372585 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.372587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.372599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.372655 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:40.373634 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.374137 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.375145 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.375438 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.375523 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.377629 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:40.377631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:40.377631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:40.377640 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:40.379494 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.379874 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.379876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.379876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.379939 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.380168 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.380169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.380169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.380313 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:40.380382 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.380608 (+ 226us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.380711 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:16:40.380908 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.371771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:54805)
0924 00:16:40.371935 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.371968 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.372470 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.372731 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.372735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.373349 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:40.373534 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.373539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.374239 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.374240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.375054 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.375055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.375533 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.375534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.375658 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.376138 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:40.376146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:40.377535 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:40.379581 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:40.379588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:40.379590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:40.379795 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:40.380007 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:40.380008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:40.380008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:40.380104 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:40.380376 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:40.380380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:40.380482 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.380558 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.380668 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260924 00:16:40.381450 15848 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42855
I20260924 00:16:40.381567 15848 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:40.381764 15848 heartbeater.cc:511] Master 127.1.44.190:42855 requested a full tablet report, sending...
I20260924 00:16:40.382126 15522 ts_manager.cc:194] Registered new tserver with Master: 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723)
I20260924 00:16:40.382486 15522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:54805
W20260924 00:16:40.457770 15851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:40.458033 15851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:40.458060 15851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:40.459235 15851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:40.459285 15851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:40.459311 15851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:16:40.460712 15851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:38081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=42329
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:40.460932 15851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:40.461143 15851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:40.466042 15857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.466084 15856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.466110 15851 server_base.cc:1034] running on GCE node
W20260924 00:16:40.466096 15859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.466463 15851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:40.466665 15851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:40.467798 15851 hybrid_clock.cc:648] HybridClock initialized: now 1790209000467789 us; error 26 us; skew 500 ppm
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:40.470124 15851 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:16:40.470548 15851 webserver.cc:460] Webserver started at http://127.1.44.131:42329/ using document root <none> and password file <none>
I20260924 00:16:40.470759 15851 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:40.470834 15851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:40.472209 15851 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.472899 15866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.473101 15851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.473199 15851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "260b0cfbc64c433e9c22e210c321bbdb"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "8d6823fcb910ee92be60f9060893ed79"
server_key_iv: "74ac25e97bc35d4cb13c26c7d2f728d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:40.473552 15851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:40.481205 15851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:40.483063 15851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:40.483263 15851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:40.483500 15851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:40.483791 15851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:40.483842 15851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.483880 15851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:40.483906 15851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.489326 15851 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:38081
I20260924 00:16:40.489416 15979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:38081 every 8 connection(s)
I20260924 00:16:40.489722 15851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:16:40.492229 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 15851
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:40.499982 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.492635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.1.44.190:42855)
0924 00:16:40.492735 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.492736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.492745 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:16:40.492807 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.492807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.492815 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.492863 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:40.493466 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.494186 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.495085 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.495357 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.495430 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.496757 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:40.496758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:40.496759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:40.496766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:40.498686 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.499035 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.499036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.499036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.499118 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.499367 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.499368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.499368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.499496 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:40.499677 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.499714 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.499789 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260924 00:16:40.500052 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.490410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44007 (local address 127.1.44.190:42855)
0924 00:16:40.490469 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.490471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.490751 (+ 280us) server_negotiation.cc:407] Connection header received
0924 00:16:40.491260 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.491261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.491276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.491344 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:40.492326 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.493073 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.494412 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.494702 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.494773 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.497114 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:40.497115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:40.497116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:40.497123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:40.498643 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.499033 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.499034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.499035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.499104 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:40.499375 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:40.499376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:40.499376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:40.499511 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:40.499584 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.499858 (+ 274us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.499929 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 00:16:40.500193 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.490409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:44007)
0924 00:16:40.490632 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.490666 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.491107 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.491397 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.491400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.491946 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:40.492219 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.492226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.493160 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.493161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.494317 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.494319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.494785 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.494786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.494880 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.495410 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:40.495419 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:40.497021 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:40.498722 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:40.498728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:40.498730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:40.498948 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:40.499175 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:40.499176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:40.499176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:40.499306 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:40.499588 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:40.499594 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:40.499722 (+ 128us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.499822 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.499970 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260924 00:16:40.500641 15980 heartbeater.cc:347] Connected to a master server at 127.1.44.190:42855
I20260924 00:16:40.500741 15980 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:40.500959 15980 heartbeater.cc:511] Master 127.1.44.190:42855 requested a full tablet report, sending...
I20260924 00:16:40.501298 15522 ts_manager.cc:194] Registered new tserver with Master: 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
I20260924 00:16:40.501653 15522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:44007
I20260924 00:16:40.511869 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:16:40.514546 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.512224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.1.44.190:42855)
0924 00:16:40.512276 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.512278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.512364 (+ 86us) server_negotiation.cc:407] Connection header received
0924 00:16:40.512417 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.512417 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.512445 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.512516 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:16:40.512614 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.513061 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.513843 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.514035 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.514089 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.514206 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:16:40.514290 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:16:40.514347 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.514412 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.514429 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 00:16:40.514808 15522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:16:40.516353 15522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:16:40.518296 15644 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdba1608c384742a1107048a569541d (DEFAULT_TABLE table=test-table [id=e1f36c98017b41bf8e58fbfe763c5776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:40.518574 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdba1608c384742a1107048a569541d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:40.520140 15602 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Bootstrap starting.
I20260924 00:16:40.520470 15602 tablet_bootstrap.cc:654] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:40.521035 15602 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: No bootstrap required, opened a new log
I20260924 00:16:40.521090 15602 ts_tablet_manager.cc:1397] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 00:16:40.521189 15602 raft_consensus.cc:348] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.521237 15602 raft_consensus.cc:374] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:40.521291 15602 raft_consensus.cc:729] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de90510f61444ecbb91f3949ac022ced, State: Initialized, Role: FOLLOWER
I20260924 00:16:40.521338 15602 consensus_queue.cc:260] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.521426 15602 ts_tablet_manager.cc:1428] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.522250 15986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.518067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:38081 (local address 127.0.0.1:45164)
0924 00:16:40.518424 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.518436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.518501 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.518675 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.518676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.518678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:40.518716 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.518717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.519935 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.519936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.521006 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.521009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.522019 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.522020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.522082 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.522086 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.522143 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.522180 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":78,"threads_started":1}
I20260924 00:16:40.522261 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.518250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.1.44.131:38081)
0924 00:16:40.518423 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.518428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.518492 (+ 64us) server_negotiation.cc:407] Connection header received
0924 00:16:40.518565 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.518567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.518584 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.518670 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:40.518782 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.519824 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.521086 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.521871 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.521934 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.521987 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.522172 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.522205 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260924 00:16:40.523064 15914 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdba1608c384742a1107048a569541d (DEFAULT_TABLE table=test-table [id=e1f36c98017b41bf8e58fbfe763c5776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:40.523411 15914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdba1608c384742a1107048a569541d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:40.523873 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.518058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37723 (local address 127.0.0.1:45456)
0924 00:16:40.518145 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.518157 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.518196 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.520355 (+ 2159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.520356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.520357 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:40.520392 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.520393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.521694 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.521695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.522808 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.522810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.523710 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.523711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.523756 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.523758 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.523802 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.523830 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 00:16:40.523950 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.518149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.1.44.130:37723)
0924 00:16:40.520201 (+ 2052us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.520206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.520222 (+ 16us) server_negotiation.cc:407] Connection header received
0924 00:16:40.520260 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.520262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.520278 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.520347 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:40.520445 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.521575 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.522923 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.523635 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.523710 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.523763 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.523827 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.523869 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2016,"thread_start_us":89,"threads_started":1}
I20260924 00:16:40.524546 15782 tablet_service.cc:1467] Processing CreateTablet for tablet 5cdba1608c384742a1107048a569541d (DEFAULT_TABLE table=test-table [id=e1f36c98017b41bf8e58fbfe763c5776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:40.524863 15782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cdba1608c384742a1107048a569541d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:40.526876 15990 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Bootstrap starting.
I20260924 00:16:40.527581 15990 tablet_bootstrap.cc:654] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:40.528029 15991 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Bootstrap starting.
I20260924 00:16:40.528119 15990 log.cc:824] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:40.528694 15991 tablet_bootstrap.cc:654] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:40.529024 15990 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: No bootstrap required, opened a new log
I20260924 00:16:40.529109 15990 ts_tablet_manager.cc:1397] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:40.529213 15991 log.cc:824] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:40.529915 15991 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: No bootstrap required, opened a new log
I20260924 00:16:40.529968 15991 ts_tablet_manager.cc:1397] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:16:40.530895 15990 raft_consensus.cc:348] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.531056 15990 raft_consensus.cc:374] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:40.531102 15990 raft_consensus.cc:729] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 260b0cfbc64c433e9c22e210c321bbdb, State: Initialized, Role: FOLLOWER
I20260924 00:16:40.531244 15990 consensus_queue.cc:260] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.531540 15990 ts_tablet_manager.cc:1428] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260924 00:16:40.531546 15980 heartbeater.cc:503] Master 127.1.44.190:42855 was elected leader, sending a full tablet report...
I20260924 00:16:40.531662 15991 raft_consensus.cc:348] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.531859 15991 raft_consensus.cc:374] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:40.531900 15991 raft_consensus.cc:729] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1ddd082abd48b686827116192d1876, State: Initialized, Role: FOLLOWER
I20260924 00:16:40.532011 15991 consensus_queue.cc:260] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.532270 15991 ts_tablet_manager.cc:1428] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:16:40.532279 15848 heartbeater.cc:503] Master 127.1.44.190:42855 was elected leader, sending a full tablet report...
W20260924 00:16:40.621745 15849 tablet.cc:2369] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:16:40.700922 15716 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:40.701061 15716 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.701383 15716 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
I20260924 00:16:40.704365 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.701622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:51679 (local address 127.1.44.131:38081)
0924 00:16:40.701686 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.701688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.701806 (+ 118us) server_negotiation.cc:407] Connection header received
0924 00:16:40.701873 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.701874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.701884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.701956 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:40.702112 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.702598 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.703630 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.703997 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.704082 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.704091 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.704249 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.704290 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:16:40.704377 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.701525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37723 (local address 127.1.44.129:57295)
0924 00:16:40.701581 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.701594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.701641 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.701811 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.701812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.701814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:40.702139 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.702141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.702768 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.702770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.703538 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.703540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.704099 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.704100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.704181 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.704184 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.704248 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.704300 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":2}
I20260924 00:16:40.704371 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.701640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:57295 (local address 127.1.44.130:37723)
0924 00:16:40.701686 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.701688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.701696 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:16:40.701722 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.701722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.701734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.701807 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:40.702223 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.702685 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.703636 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.704013 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.704102 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.704111 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.704267 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.704306 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:16:40.704396 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.701541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:38081 (local address 127.1.44.129:51679)
0924 00:16:40.701721 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.701730 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:40.701813 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:40.701959 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:40.701960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:40.701961 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:40.702041 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.702043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.702682 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.702683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.703540 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:40.703543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.704116 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.704117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:40.704179 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.704183 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:16:40.704245 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 00:16:40.704290 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":36,"thread_start_us":94,"threads_started":1}
I20260924 00:16:40.704689 15934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cdba1608c384742a1107048a569541d" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260b0cfbc64c433e9c22e210c321bbdb" is_pre_election: true
I20260924 00:16:40.704759 15802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cdba1608c384742a1107048a569541d" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1ddd082abd48b686827116192d1876" is_pre_election: true
I20260924 00:16:40.704847 15934 raft_consensus.cc:2454] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 0.
I20260924 00:16:40.704950 15802 raft_consensus.cc:2454] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 0.
I20260924 00:16:40.705098 15597 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 260b0cfbc64c433e9c22e210c321bbdb, de90510f61444ecbb91f3949ac022ced; no voters:
I20260924 00:16:40.705353 15716 raft_consensus.cc:2781] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:16:40.705437 15716 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:16:40.705472 15716 raft_consensus.cc:3037] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:40.706204 15716 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.706355 15716 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 election: Requested vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
I20260924 00:16:40.706527 15934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cdba1608c384742a1107048a569541d" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "260b0cfbc64c433e9c22e210c321bbdb"
I20260924 00:16:40.706538 15802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cdba1608c384742a1107048a569541d" candidate_uuid: "de90510f61444ecbb91f3949ac022ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1ddd082abd48b686827116192d1876"
I20260924 00:16:40.706599 15934 raft_consensus.cc:3037] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:40.706624 15802 raft_consensus.cc:3037] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:40.707372 15934 raft_consensus.cc:2454] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 1.
I20260924 00:16:40.707511 15802 raft_consensus.cc:2454] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de90510f61444ecbb91f3949ac022ced in term 1.
I20260924 00:16:40.707701 15597 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 260b0cfbc64c433e9c22e210c321bbdb, de90510f61444ecbb91f3949ac022ced; no voters:
I20260924 00:16:40.707830 15716 raft_consensus.cc:2781] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:40.708048 15716 raft_consensus.cc:686] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 LEADER]: Becoming Leader. State: Replica: de90510f61444ecbb91f3949ac022ced, State: Running, Role: LEADER
I20260924 00:16:40.708128 15716 consensus_queue.cc:237] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:40.708860 15522 catalog_manager.cc:5495] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced reported cstate change: term changed from 0 to 1, leader changed from <none> to de90510f61444ecbb91f3949ac022ced (127.1.44.129). New cstate: current_term: 1 leader_uuid: "de90510f61444ecbb91f3949ac022ced" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } health_report { overall_health: HEALTHY } } }
W20260924 00:16:40.740265 15981 tablet.cc:2369] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:16:40.743881 15711 tablet.cc:2369] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:16:40.748065 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.745938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.1.44.129:41629)
0924 00:16:40.746013 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:40.746014 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:40.746023 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:16:40.746054 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:40.746054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:40.746072 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:40.746132 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:16:40.746245 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.746696 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.747436 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:40.747636 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:40.747689 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:40.747813 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:16:40.747905 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:16:40.747957 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:40.747996 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 00:16:40.748013 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 00:16:40.749527 15934 raft_consensus.cc:1261] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Refusing update from remote peer de90510f61444ecbb91f3949ac022ced: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:40.749547 15802 raft_consensus.cc:1261] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Refusing update from remote peer de90510f61444ecbb91f3949ac022ced: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:40.749905 15997 consensus_queue.cc:1035] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [LEADER]: Connected to new peer: Peer: permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:40.750043 15716 consensus_queue.cc:1035] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:40.752096 15999 mvcc.cc:205] Tried to move back new op lower bound from 7332696067067432960 to 7332696066901110784. Current Snapshot: MvccSnapshot[applied={T|T < 7332696067067432960}]
I20260924 00:16:40.752980 16000 mvcc.cc:205] Tried to move back new op lower bound from 7332696067067432960 to 7332696066901110784. Current Snapshot: MvccSnapshot[applied={T|T < 7332696067067432960}]
I20260924 00:16:40.753527 16001 mvcc.cc:205] Tried to move back new op lower bound from 7332696067067432960 to 7332696066901110784. Current Snapshot: MvccSnapshot[applied={T|T < 7332696067067432960 or (T in {7332696067067432960})}]
I20260924 00:16:40.754385 15521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:40.754484 15521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:40.755184 15521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Sending DeleteTablet for 3 replicas of tablet 5cdba1608c384742a1107048a569541d
I20260924 00:16:40.755507 15914 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cdba1608c384742a1107048a569541d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:40 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45164
I20260924 00:16:40.755527 15644 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cdba1608c384742a1107048a569541d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:40 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:53732
I20260924 00:16:40.755558 15782 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cdba1608c384742a1107048a569541d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:40 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45456
I20260924 00:16:40.755607 15719 tablet_replica.cc:330] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: stopping tablet replica
I20260924 00:16:40.755647 15719 raft_consensus.cc:2229] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:16:40.755717 15719 raft_consensus.cc:2258] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:40.755746 16009 tablet_replica.cc:330] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: stopping tablet replica
I20260924 00:16:40.755857 16009 raft_consensus.cc:2229] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:40.755966 16009 raft_consensus.cc:2258] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:40.756019 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15560
I20260924 00:16:40.756129 16010 tablet_replica.cc:330] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: stopping tablet replica
I20260924 00:16:40.756211 16010 raft_consensus.cc:2229] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:40.756299 16010 raft_consensus.cc:2258] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:40.757755 16010 ts_tablet_manager.cc:1905] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:40.758899 16009 ts_tablet_manager.cc:1905] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:40.759341 16010 ts_tablet_manager.cc:1918] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:40.759419 16010 log.cc:1196] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/5cdba1608c384742a1107048a569541d
I20260924 00:16:40.759696 16010 ts_tablet_manager.cc:1939] T 5cdba1608c384742a1107048a569541d P 8a1ddd082abd48b686827116192d1876: Deleting consensus metadata
I20260924 00:16:40.760138 15509 catalog_manager.cc:4841] TS 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): tablet 5cdba1608c384742a1107048a569541d (table test-table [id=e1f36c98017b41bf8e58fbfe763c5776]) successfully deleted
I20260924 00:16:40.760632 16009 ts_tablet_manager.cc:1918] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:40.760716 16009 log.cc:1196] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/5cdba1608c384742a1107048a569541d
I20260924 00:16:40.760962 16009 ts_tablet_manager.cc:1939] T 5cdba1608c384742a1107048a569541d P 260b0cfbc64c433e9c22e210c321bbdb: Deleting consensus metadata
I20260924 00:16:40.761336 15507 catalog_manager.cc:4841] TS 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): tablet 5cdba1608c384742a1107048a569541d (table test-table [id=e1f36c98017b41bf8e58fbfe763c5776]) successfully deleted
W20260924 00:16:40.762303 15508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:41629 (error 108)
I20260924 00:16:40.762598 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15715
I20260924 00:16:40.762768 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:40.762636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41629 (local address 127.0.0.1:53744)
0924 00:16:40.762700 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:40.762733 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 00:16:40.762864 15508 catalog_manager.cc:4584] TS de90510f61444ecbb91f3949ac022ced (127.1.44.129:41629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5cdba1608c384742a1107048a569541d: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41629: connect: Connection refused (error 111)
I20260924 00:16:40.767889 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15851
I20260924 00:16:40.772184 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 15489
I20260924 00:16:40.777266 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42855
--webserver_interface=127.1.44.190
--webserver_port=34855
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:40.856000 16012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:40.856190 16012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:40.856213 16012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:40.857332 16012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:40.857373 16012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:40.857388 16012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:40.857426 16012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:40.857457 16012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:40.858883 16012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:42855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=34855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:40.859117 16012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:40.859370 16012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:40.864161 16020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:40.864153 16018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.864251 16012 server_base.cc:1034] running on GCE node
W20260924 00:16:40.864152 16017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:40.864579 16012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:40.864858 16012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:40.866021 16012 hybrid_clock.cc:648] HybridClock initialized: now 1790209000866017 us; error 33 us; skew 500 ppm
Sep 24 00:16:40 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209000, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:40.868328 16012 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:40.868736 16012 webserver.cc:460] Webserver started at http://127.1.44.190:34855/ using document root <none> and password file <none>
I20260924 00:16:40.868947 16012 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:40.869022 16012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:40.870311 16012 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:40.870932 16027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:40.871102 16012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:40.871197 16012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "e2299082a9ed49a4b986eb1419947344"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c80f93bba8b0261df1cf5f9365e079c0"
server_key_iv: "212714c81c1424d12641729aabf7021b"
server_key_version: "encryptionkey@0"
I20260924 00:16:40.871563 16012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:40.888070 16012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:40.889775 16012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:40.889930 16012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:40.893726 16012 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:42855
I20260924 00:16:40.893759 16079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:42855 every 8 connection(s)
I20260924 00:16:40.894109 16012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:40.895591 16080 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:16:40.896219 16080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap starting.
I20260924 00:16:40.902016 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16012
I20260924 00:16:40.902290 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:41629
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:42855
--webserver_port=41395
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:40.904147 16080 log.cc:824] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:40.906194 16080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:16:40.906473 16080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344: Bootstrap complete.
I20260924 00:16:40.908609 16080 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.908838 16080 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2299082a9ed49a4b986eb1419947344, State: Initialized, Role: FOLLOWER
I20260924 00:16:40.909014 16080 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.909096 16080 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:40.909147 16080 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:40.909197 16080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:16:40.910115 16080 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.910228 16080 leader_election.cc:302] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2299082a9ed49a4b986eb1419947344; no voters:
I20260924 00:16:40.910413 16080 leader_election.cc:288] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 00:16:40.910465 16084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 00:16:40.910583 16084 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [term 3 LEADER]: Becoming Leader. State: Replica: e2299082a9ed49a4b986eb1419947344, State: Running, Role: LEADER
I20260924 00:16:40.910671 16084 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } }
I20260924 00:16:40.910677 16080 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:40.910955 16086 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e2299082a9ed49a4b986eb1419947344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } } }
I20260924 00:16:40.911063 16086 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:40.911176 16085 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2299082a9ed49a4b986eb1419947344. Latest consensus state: current_term: 3 leader_uuid: "e2299082a9ed49a4b986eb1419947344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2299082a9ed49a4b986eb1419947344" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 42855 } } }
I20260924 00:16:40.911267 16085 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2299082a9ed49a4b986eb1419947344 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:40.911753 16092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:40.912514 16092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:40.912771 16092 catalog_manager.cc:1259] Loaded cluster ID: 9cc695db42f64f9cbb3a6d95b1455d29
I20260924 00:16:40.912842 16092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 00:16:40.913100 16092 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f33f47289ad google::LogMessage::Fail() at ??:0
@ 0x7f33f472b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f33f47284de google::LogMessage::Flush() at ??:0
@ 0x7f33f4729a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f33fc64d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f33fc6568b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f33f65f60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f33f65eb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f33f744f6db start_thread at ??:0
@ 0x7f33f322661f clone at ??:0
W20260924 00:16:41.020699 16082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:41.021008 16082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:41.021049 16082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:41.022796 16082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:41.022858 16082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:41.022899 16082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:16:41.025120 16082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:41629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=41395
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:41.025369 16082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:41.025679 16082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:41.030886 16116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:41.030901 16117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:41.031009 16119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:41.031315 16082 server_base.cc:1034] running on GCE node
I20260924 00:16:41.031522 16082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:41.031780 16082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:41.032948 16082 hybrid_clock.cc:648] HybridClock initialized: now 1790209001032935 us; error 30 us; skew 500 ppm
Sep 24 00:16:41 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209001, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:41.035507 16082 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:16:41.035910 16082 webserver.cc:460] Webserver started at http://127.1.44.129:41395/ using document root <none> and password file <none>
I20260924 00:16:41.036144 16082 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:41.036209 16082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:41.037642 16082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:41.038362 16126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.038545 16082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:41.038664 16082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "de90510f61444ecbb91f3949ac022ced"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "696862627848222aa8ca4b489808a4bf"
server_key_iv: "a3cb1ecd77867f5295969999373971f0"
server_key_version: "encryptionkey@0"
I20260924 00:16:41.039003 16082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:41.056214 16082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:41.058326 16082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:41.058478 16082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:41.058697 16082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:41.059211 16133 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 00:16:41.061039 16082 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 00:16:41.061100 16082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 00:16:41.061138 16082 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 00:16:41.061731 16082 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 00:16:41.061782 16082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 00:16:41.061846 16133 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Bootstrap starting.
I20260924 00:16:41.068617 16082 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:41629
I20260924 00:16:41.069028 16082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:16:41.072005 16240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:41629 every 8 connection(s)
I20260924 00:16:41.072535 16133 log.cc:824] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:41.075619 16133 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:16:41.075913 16133 tablet_bootstrap.cc:492] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Bootstrap complete.
I20260924 00:16:41.077102 16133 ts_tablet_manager.cc:1397] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260924 00:16:41.077394 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16082
I20260924 00:16:41.077646 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:37723
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:42855
--webserver_port=35019
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:16:41.078501 16133 raft_consensus.cc:348] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:41.078805 16133 raft_consensus.cc:729] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de90510f61444ecbb91f3949ac022ced, State: Initialized, Role: FOLLOWER
I20260924 00:16:41.078928 16133 consensus_queue.cc:260] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:41.079170 16133 ts_tablet_manager.cc:1428] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260924 00:16:41.125056 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:16:41.072705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:56721)
0924 00:16:41.072890 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.072936 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:41.073491 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:41.124865 (+ 51374us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
W20260924 00:16:41.125281 16241 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:42855 (0 consecutive failures): Network error: Failed to ping master at 127.1.44.190:42855: Client connection negotiation failed: client connection to 127.1.44.190:42855: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:16:41.125643 16244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.125510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:33951)
0924 00:16:41.125565 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.125612 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 00:16:41.126036 16244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.125948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:44129)
0924 00:16:41.125998 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.126021 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 00:16:41.126171 16241 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:16:41.161047 16245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:41.161235 16245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:41.161260 16245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:41.162393 16245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:41.162436 16245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:41.162461 16245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:16:41.163882 16245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:37723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=35019
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:41.164114 16245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:41.164347 16245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:41.169242 16254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:41.169252 16251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:41.169355 16245 server_base.cc:1034] running on GCE node
W20260924 00:16:41.169242 16252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:41.169693 16245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:41.169935 16245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:41.171070 16245 hybrid_clock.cc:648] HybridClock initialized: now 1790209001171058 us; error 43 us; skew 500 ppm
Sep 24 00:16:41 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209001, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:41.173434 16245 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:16:41.173872 16245 webserver.cc:460] Webserver started at http://127.1.44.130:35019/ using document root <none> and password file <none>
I20260924 00:16:41.174109 16245 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:41.174176 16245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:41.175494 16245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:41.176080 16261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.176266 16245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:41.176381 16245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "8a1ddd082abd48b686827116192d1876"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "c2df08da05cae1bfd4f10a30f11714b4"
server_key_iv: "26f1d653ced83fd2f06a4d31288d43d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:41.176692 16245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:41.191323 16245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:41.193078 16245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:41.193233 16245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:41.193449 16245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:41.193738 16245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:41.193790 16245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.193823 16245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:41.193850 16245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.199455 16245 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:37723
I20260924 00:16:41.199509 16374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:37723 every 8 connection(s)
I20260924 00:16:41.199903 16245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:16:41.200883 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.200557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:35597)
0924 00:16:41.200693 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.200754 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
W20260924 00:16:41.201274 16375 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:42855 (0 consecutive failures): Network error: Failed to ping master at 127.1.44.190:42855: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
I20260924 00:16:41.201537 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.201453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:49293)
0924 00:16:41.201495 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.201521 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 00:16:41.201884 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.201790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:33129)
0924 00:16:41.201836 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.201862 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 00:16:41.202004 16375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:16:41.203339 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16245
I20260924 00:16:41.203614 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:38081
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:42855
--webserver_port=42329
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:36271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:41.283145 16378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:41.283463 16378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:41.283507 16378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:41.284634 16378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:41.284672 16378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:41.284698 16378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:16:41.286051 16378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:36271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:38081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=42329
--tserver_master_addrs=127.1.44.190:42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:41.286216 16378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:41.286374 16378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:41.291085 16384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:41.291096 16383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:41.291096 16386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:41.291378 16378 server_base.cc:1034] running on GCE node
I20260924 00:16:41.291546 16378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:41.291726 16378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:41.292860 16378 hybrid_clock.cc:648] HybridClock initialized: now 1790209001292848 us; error 26 us; skew 500 ppm
Sep 24 00:16:41 dist-test-slave-cm7b krb5kdc[14942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209001, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:41.295078 16378 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:16:41.295565 16378 webserver.cc:460] Webserver started at http://127.1.44.131:42329/ using document root <none> and password file <none>
I20260924 00:16:41.295790 16378 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:41.295856 16378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:41.297106 16378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:16:41.297675 16393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.297857 16378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:16:41.297981 16378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "260b0cfbc64c433e9c22e210c321bbdb"
format_stamp: "Formatted at 2026-09-24 00:16:39 on dist-test-slave-cm7b"
server_key: "8d6823fcb910ee92be60f9060893ed79"
server_key_iv: "74ac25e97bc35d4cb13c26c7d2f728d8"
server_key_version: "encryptionkey@0"
I20260924 00:16:41.298296 16378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:41.310598 16378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:41.312522 16378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:41.312678 16378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:41.312892 16378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:41.313177 16378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:41.313228 16378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.313261 16378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:41.313289 16378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:41.318924 16378 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:38081
I20260924 00:16:41.318975 16506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:38081 every 8 connection(s)
I20260924 00:16:41.319355 16378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:16:41.320302 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.319948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:59749)
0924 00:16:41.320102 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.320173 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
W20260924 00:16:41.320701 16507 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:42855 (0 consecutive failures): Network error: Failed to ping master at 127.1.44.190:42855: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
I20260924 00:16:41.320959 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.320878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:49583)
0924 00:16:41.320919 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.320943 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 00:16:41.321237 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:41.321169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:47897)
0924 00:16:41.321205 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:41.321225 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 00:16:41.321355 16507 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:16:41.327945 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16378
I20260924 00:16:42.126802 16511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.126554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:40147)
0924 00:16:42.126701 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:42.126737 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260924 00:16:42.202576 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.202352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:54373)
0924 00:16:42.202490 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:42.202527 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260924 00:16:42.321812 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.321629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:39445)
0924 00:16:42.321739 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:42.321771 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260924 00:16:42.698458 16514 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:42.698621 16514 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:42.698984 16514 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
Sep 24 00:16:42 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209001, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.131@KRBTEST.COM
Sep 24 00:16:42 dist-test-slave-cm7b krb5kdc[14942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209001, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.130@KRBTEST.COM
I20260924 00:16:42.709591 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.699306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:60515 (local address 127.1.44.131:38081)
0924 00:16:42.699464 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:42.699473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:42.699520 (+ 47us) server_negotiation.cc:407] Connection header received
0924 00:16:42.699986 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:42.699988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:42.700025 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:42.700164 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:42.700945 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.701875 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.702874 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.703413 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.703543 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:42.705789 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:42.705793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:42.705793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:42.705806 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:42.707718 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708191 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:42.708193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:42.708194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:42.708304 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708608 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:42.708609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:42.708609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:42.708901 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:42.708986 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:42.709340 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 00:16:42.709470 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260924 00:16:42.709591 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.699296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:60985 (local address 127.1.44.130:37723)
0924 00:16:42.699449 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:42.699463 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:42.699498 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:16:42.699962 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:42.699964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:42.700020 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:42.700164 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:42.700906 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.701902 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.702835 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.703414 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.703549 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:42.706147 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:42.706149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:42.706150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:42.706158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:42.707944 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708296 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:42.708298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:42.708298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:42.708395 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708608 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:42.708609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:42.708609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:42.708901 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:42.708986 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:42.709340 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 00:16:42.709470 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":48,"threads_started":1}
I20260924 00:16:42.709610 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.699146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:38081 (local address 127.1.44.129:60515)
0924 00:16:42.699445 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:42.699456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:42.699535 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:42.700204 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:42.700206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:42.700647 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:42.700803 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:42.700808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.701964 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.701965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:42.702775 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:42.702778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.703516 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.703517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:42.703643 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:42.704218 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:42.704227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:42.705679 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:42.707809 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:42.707815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:42.707816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:42.708099 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:42.708373 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:42.708374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:42.708532 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:42.708986 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:42.708993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:42.709151 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 00:16:42.709271 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 00:16:42.709481 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":61,"thread_start_us":83,"threads_started":1}
I20260924 00:16:42.709789 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:42.699122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37723 (local address 127.1.44.129:60985)
0924 00:16:42.699293 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:42.699302 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:42.699353 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:42.700203 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:42.700206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:42.700607 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:42.700802 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:42.700808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.702001 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.702002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:42.702743 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:42.702745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:42.703520 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:42.703521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:42.703639 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:42.704287 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:42.704292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:42.706040 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:42.708027 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708027 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:42.708028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:42.708227 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:42.708462 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:42.708462 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:42.708462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:42.708532 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:42.708986 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:42.708993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:42.709151 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 00:16:42.709271 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 00:16:42.709542 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":168,"thread_start_us":103,"threads_started":1}
W20260924 00:16:42.710533 16130 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
W20260924 00:16:42.710994 16128 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
I20260924 00:16:42.711048 16128 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de90510f61444ecbb91f3949ac022ced; no voters: 260b0cfbc64c433e9c22e210c321bbdb, 8a1ddd082abd48b686827116192d1876
I20260924 00:16:42.711177 16514 raft_consensus.cc:2726] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:16:43.127594 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:43.127387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:57159)
0924 00:16:43.127486 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:43.127535 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 00:16:43.203286 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:43.202996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:49517)
0924 00:16:43.203136 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:43.203171 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260924 00:16:43.322428 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:43.322228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:40247)
0924 00:16:43.322357 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:43.322391 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260924 00:16:44.128278 16521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:44.128061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:49983)
0924 00:16:44.128191 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:44.128230 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260924 00:16:44.203910 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:44.203724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:38111)
0924 00:16:44.203839 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:44.203872 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260924 00:16:44.323143 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:44.322941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:55991)
0924 00:16:44.323074 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:44.323103 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260924 00:16:44.584172 16524 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:44.584259 16524 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:44.584517 16524 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
W20260924 00:16:44.584858 16130 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
W20260924 00:16:44.584940 16128 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
I20260924 00:16:44.585044 16128 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de90510f61444ecbb91f3949ac022ced; no voters: 260b0cfbc64c433e9c22e210c321bbdb, 8a1ddd082abd48b686827116192d1876
I20260924 00:16:44.585117 16524 raft_consensus.cc:2726] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:16:45.129086 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:45.128814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:41069)
0924 00:16:45.128981 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:45.129035 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":64,"threads_started":1}
I20260924 00:16:45.204639 16526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:45.204391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:48373)
0924 00:16:45.204520 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:45.204566 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260924 00:16:45.323828 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:45.323619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:37495)
0924 00:16:45.323749 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:45.323789 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260924 00:16:46.129871 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:46.129637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:47999)
0924 00:16:46.129791 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:46.129826 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":63,"threads_started":1}
I20260924 00:16:46.205284 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:46.205072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:57979)
0924 00:16:46.205210 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:46.205245 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":57,"threads_started":1}
I20260924 00:16:46.324384 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:46.324203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:57375)
0924 00:16:46.324319 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:46.324350 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260924 00:16:47.130642 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:47.130386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:60347)
0924 00:16:47.130556 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:47.130594 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260924 00:16:47.205927 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:47.205733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:57151)
0924 00:16:47.205858 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:47.205889 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260924 00:16:47.324963 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:47.324792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:50233)
0924 00:16:47.324898 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:47.324930 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260924 00:16:47.696331 16534 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:47.696463 16534 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:47.696635 16534 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
W20260924 00:16:47.696985 16128 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
W20260924 00:16:47.697077 16130 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
I20260924 00:16:47.697140 16130 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de90510f61444ecbb91f3949ac022ced; no voters: 260b0cfbc64c433e9c22e210c321bbdb, 8a1ddd082abd48b686827116192d1876
I20260924 00:16:47.697232 16534 raft_consensus.cc:2726] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:16:48.131481 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:48.131179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:42015)
0924 00:16:48.131381 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:48.131429 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":128,"threads_started":1}
I20260924 00:16:48.206568 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:48.206350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:44583)
0924 00:16:48.206482 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:48.206521 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260924 00:16:48.325567 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:48.325358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:60275)
0924 00:16:48.325485 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:48.325520 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260924 00:16:49.132192 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:49.131950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:43369)
0924 00:16:49.132100 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:49.132139 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260924 00:16:49.207397 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:49.207136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:47613)
0924 00:16:49.207321 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:49.207357 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260924 00:16:49.326293 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:49.326048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:43229)
0924 00:16:49.326191 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:49.326242 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260924 00:16:50.132941 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:50.132740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:36477)
0924 00:16:50.132864 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:50.132893 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260924 00:16:50.208098 16542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:50.207882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:44741)
0924 00:16:50.208007 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:50.208046 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260924 00:16:50.326979 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:50.326750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:53489)
0924 00:16:50.326897 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:50.326934 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260924 00:16:51.133790 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:51.133558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:34001)
0924 00:16:51.133702 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:51.133742 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260924 00:16:51.208767 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:51.208553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:37923)
0924 00:16:51.208685 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:51.208720 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260924 00:16:51.327726 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:51.327482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:45389)
0924 00:16:51.327639 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:51.327681 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
I20260924 00:16:51.376294 16547 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:51.376371 16547 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:51.376602 16547 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
W20260924 00:16:51.376976 16130 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
W20260924 00:16:51.377099 16128 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
I20260924 00:16:51.377140 16128 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de90510f61444ecbb91f3949ac022ced; no voters: 260b0cfbc64c433e9c22e210c321bbdb, 8a1ddd082abd48b686827116192d1876
I20260924 00:16:51.377223 16547 raft_consensus.cc:2726] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:16:52.134511 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:52.134302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:36043)
0924 00:16:52.134422 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:52.134464 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260924 00:16:52.209450 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:52.209261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:39961)
0924 00:16:52.209376 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:52.209408 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260924 00:16:52.328473 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:52.328255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:49483)
0924 00:16:52.328373 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:52.328414 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260924 00:16:53.135326 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:53.135036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:55237)
0924 00:16:53.135186 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:53.135280 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260924 00:16:53.210145 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:53.209910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:41753)
0924 00:16:53.210051 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:53.210100 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260924 00:16:53.329249 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:53.329020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:34413)
0924 00:16:53.329159 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:53.329203 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260924 00:16:54.136072 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:54.135846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:35967)
0924 00:16:54.135977 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:54.136022 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260924 00:16:54.210852 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:54.210624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:46479)
0924 00:16:54.210768 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:54.210807 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260924 00:16:54.329965 16556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:54.329756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:48645)
0924 00:16:54.329891 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:54.329922 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260924 00:16:54.924737 16557 raft_consensus.cc:482] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:54.924831 16557 raft_consensus.cc:504] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a1ddd082abd48b686827116192d1876" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37723 } } peers { permanent_uuid: "260b0cfbc64c433e9c22e210c321bbdb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 38081 } } peers { permanent_uuid: "de90510f61444ecbb91f3949ac022ced" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41629 } }
I20260924 00:16:54.925021 16557 leader_election.cc:288] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723), 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081)
W20260924 00:16:54.925458 16130 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a1ddd082abd48b686827116192d1876 (127.1.44.130:37723): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
W20260924 00:16:54.925530 16128 leader_election.cc:341] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 260b0cfbc64c433e9c22e210c321bbdb (127.1.44.131:38081): Not found: Tablet not found: 5cdba1608c384742a1107048a569541d
I20260924 00:16:54.925561 16128 leader_election.cc:302] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de90510f61444ecbb91f3949ac022ced; no voters: 260b0cfbc64c433e9c22e210c321bbdb, 8a1ddd082abd48b686827116192d1876
I20260924 00:16:54.925652 16557 raft_consensus.cc:2726] T 5cdba1608c384742a1107048a569541d P de90510f61444ecbb91f3949ac022ced [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:16:55.136790 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:55.136578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:48433)
0924 00:16:55.136715 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:55.136744 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":54,"threads_started":1}
I20260924 00:16:55.211668 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:55.211453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:56673)
0924 00:16:55.211585 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:55.211621 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
I20260924 00:16:55.330719 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:55.330460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:50963)
0924 00:16:55.330594 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:55.330642 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260924 00:16:56.137421 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:56.137224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.129:47375)
0924 00:16:56.137345 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:56.137376 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260924 00:16:56.212342 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:56.212149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.130:60459)
0924 00:16:56.212267 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:56.212301 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260924 00:16:56.331496 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:56.331276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:42855 (local address 127.1.44.131:38783)
0924 00:16:56.331411 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:56.331449 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:42855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260924 00:16:56.334136 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16082
I20260924 00:16:56.338757 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16245
I20260924 00:16:56.342962 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16378
2026-09-24T00:16:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18391 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:16:56 dist-test-slave-cm7b krb5kdc[16566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:16:56 dist-test-slave-cm7b krb5kdc[16566](info): set up 2 sockets
Sep 24 00:16:56 dist-test-slave-cm7b krb5kdc[16566](info): commencing operation
krb5kdc: starting...
W20260924 00:16:57.367430 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:16:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:16:57.482394 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35641
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:57.560325 16579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:57.560550 16579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:57.560585 16579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:57.561765 16579 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 00:16:57.561831 16579 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 00:16:57.561852 16579 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 00:16:57.561872 16579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:16:57.561903 16579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:57.561930 16579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:57.561957 16579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:16:57.561980 16579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:16:57.563447 16579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:35641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:57.563706 16579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:57.563930 16579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:57.568648 16585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:57.568660 16587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:57.568660 16584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:57.568946 16579 server_base.cc:1034] running on GCE node
I20260924 00:16:57.569187 16579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:57.569456 16579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:57.570595 16579 hybrid_clock.cc:648] HybridClock initialized: now 1790209017570563 us; error 47 us; skew 500 ppm
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:57.572901 16579 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:16:57.573362 16579 webserver.cc:460] Webserver started at http://127.1.44.190:38109/ using document root <none> and password file <none>
I20260924 00:16:57.573586 16579 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:57.573652 16579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:57.573801 16579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:57.574791 16579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "d0c9f267f7e04239ae529d1bbf20fb92"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "9afe93a038dfe4b9abcb6a2a6432fd52"
server_key_iv: "60693015b8bae3626271a5b6a1e16c8f"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.575142 16579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "d0c9f267f7e04239ae529d1bbf20fb92"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "9afe93a038dfe4b9abcb6a2a6432fd52"
server_key_iv: "60693015b8bae3626271a5b6a1e16c8f"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.576462 16579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:16:57.577195 16594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.577373 16579 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:16:57.577466 16579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "d0c9f267f7e04239ae529d1bbf20fb92"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "9afe93a038dfe4b9abcb6a2a6432fd52"
server_key_iv: "60693015b8bae3626271a5b6a1e16c8f"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.577545 16579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:57.612691 16579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:57.614717 16579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:57.614886 16579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:57.618909 16579 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:35641
I20260924 00:16:57.618930 16646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:35641 every 8 connection(s)
I20260924 00:16:57.619302 16579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:16:57.620148 16647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:57.622283 16647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap starting.
I20260924 00:16:57.622804 16647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:57.623034 16647 log.cc:824] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:57.623629 16647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: No bootstrap required, opened a new log
I20260924 00:16:57.625092 16647 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:16:57.625218 16647 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:57.625239 16647 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c9f267f7e04239ae529d1bbf20fb92, State: Initialized, Role: FOLLOWER
I20260924 00:16:57.625379 16647 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:16:57.625461 16647 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:16:57.625494 16647 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:16:57.625526 16647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:57.625984 16647 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:16:57.626061 16647 leader_election.cc:302] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c9f267f7e04239ae529d1bbf20fb92; no voters:
I20260924 00:16:57.626186 16647 leader_election.cc:288] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:16:57.626243 16650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:57.626422 16650 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 LEADER]: Becoming Leader. State: Replica: d0c9f267f7e04239ae529d1bbf20fb92, State: Running, Role: LEADER
I20260924 00:16:57.626433 16647 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:16:57.626528 16650 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:16:57.626859 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16579
I20260924 00:16:57.626963 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:16:57.627154 16651 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } } }
I20260924 00:16:57.627305 1202 external_mini_cluster.cc:1428] Setting key b0d4b98a12f5ce9381e140004e18d778
I20260924 00:16:57.627307 16651 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:57.627166 16652 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c9f267f7e04239ae529d1bbf20fb92. Latest consensus state: current_term: 1 leader_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } } }
I20260924 00:16:57.627353 16652 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: This master's current role is: LEADER
I20260924 00:16:57.627499 16659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:16:57.628034 16659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:16:57.630452 16659 catalog_manager.cc:1347] Generated new cluster ID: 71abef1b8fb04329a9a5bc57803e7de3
I20260924 00:16:57.630504 16659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:57.639034 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:57.627929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.1.44.190:35641)
0924 00:16:57.628416 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:57.628426 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:57.628458 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:16:57.628568 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:57.628576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:57.628629 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:57.628778 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:57.629516 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.630729 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.631841 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.632437 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.632530 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:57.635315 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:57.635320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:57.635321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:57.635331 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:57.637412 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:57.637791 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:57.637794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:57.637795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:57.637900 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:57.638112 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:57.638112 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:57.638113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:57.638425 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:57.638570 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:57.638677 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 00:16:57.638819 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":72,"threads_started":1}
I20260924 00:16:57.639590 16659 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:16:57.640278 16659 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:16:57.644671 16659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Generated new TSK 0
I20260924 00:16:57.644817 16659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:16:57.670671 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35641
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:57.753991 16673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:57.754205 16673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:57.754244 16673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:57.755435 16673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:57.755494 16673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:57.755539 16673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:16:57.756958 16673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:57.757201 16673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:57.757426 16673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:57.763084 16678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:57.763055 16679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:57.763253 16673 server_base.cc:1034] running on GCE node
W20260924 00:16:57.763324 16681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:57.763525 16673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:57.763801 16673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:57.764968 16673 hybrid_clock.cc:648] HybridClock initialized: now 1790209017764948 us; error 30 us; skew 500 ppm
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:57.767393 16673 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:16:57.767804 16673 webserver.cc:460] Webserver started at http://127.1.44.129:44129/ using document root <none> and password file <none>
I20260924 00:16:57.768062 16673 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:57.768127 16673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:57.768267 16673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:57.769275 16673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "d51e1904fd0d44c9bf5b5e5093f942bc"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d95f6564f6bc9adcde086f7f0e72d070"
server_key_iv: "97f776c05ed0013f647ec5e47c84fd99"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.769622 16673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "d51e1904fd0d44c9bf5b5e5093f942bc"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d95f6564f6bc9adcde086f7f0e72d070"
server_key_iv: "97f776c05ed0013f647ec5e47c84fd99"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.770898 16673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:16:57.771641 16688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.771808 16673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.771904 16673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "d51e1904fd0d44c9bf5b5e5093f942bc"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d95f6564f6bc9adcde086f7f0e72d070"
server_key_iv: "97f776c05ed0013f647ec5e47c84fd99"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.771982 16673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:57.797640 16673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:57.799527 16673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:57.799705 16673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:57.799943 16673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:57.800251 16673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:57.800305 16673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.800340 16673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:57.800367 16673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.805768 16673 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:40999
I20260924 00:16:57.805881 16801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:40999 every 8 connection(s)
I20260924 00:16:57.806164 16673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:16:57.808650 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16673
I20260924 00:16:57.808761 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:16:57.808967 1202 external_mini_cluster.cc:1428] Setting key f3754f4edc96b0f6f42245552458fa5a
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:57.816469 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:57.806870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:46415 (local address 127.1.44.190:35641)
0924 00:16:57.806954 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:57.806957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:57.807094 (+ 137us) server_negotiation.cc:407] Connection header received
0924 00:16:57.807631 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:57.807645 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:57.807660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:57.807705 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:57.808620 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.809330 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.810413 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.810754 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.810825 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:57.813348 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:57.813350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:57.813350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:57.813358 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:57.815058 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:57.815514 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:57.815515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:57.815515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:57.815578 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:57.815821 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:57.815823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:57.815823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:57.815967 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:57.816039 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:57.816282 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 00:16:57.816367 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 00:16:57.816496 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:57.806821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:46415)
0924 00:16:57.806975 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:57.807011 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:57.807546 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:57.807767 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:57.807771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:57.808330 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:57.808520 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:57.808524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.809413 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.809414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:57.810326 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:57.810329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.810847 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.810848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:57.811009 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:57.811538 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:57.811550 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:57.813255 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:57.815137 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:57.815143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:57.815145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:57.815441 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:57.815655 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:57.815656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:57.815656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:57.815750 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:57.816032 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:57.816036 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:57.816144 (+ 108us) client_negotiation.cc:769] Sending connection context
0924 00:16:57.816216 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 00:16:57.816336 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260924 00:16:57.817000 16802 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35641
I20260924 00:16:57.817095 16802 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:57.817281 16802 heartbeater.cc:511] Master 127.1.44.190:35641 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:16:57.817824 16611 ts_manager.cc:194] Registered new tserver with Master: d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999)
I20260924 00:16:57.818401 16611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:46415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:16:57.834650 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35641
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:57.914345 16807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:57.914569 16807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:57.914606 16807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:57.915832 16807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:57.915895 16807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:57.915937 16807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:16:57.917364 16807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:57.917611 16807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:57.917843 16807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:57.922643 16815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:57.922653 16813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:57.922662 16812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:57.923050 16807 server_base.cc:1034] running on GCE node
I20260924 00:16:57.923264 16807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:57.923480 16807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:57.924615 16807 hybrid_clock.cc:648] HybridClock initialized: now 1790209017924597 us; error 33 us; skew 500 ppm
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:57.926981 16807 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:16:57.927578 16807 webserver.cc:460] Webserver started at http://127.1.44.130:45853/ using document root <none> and password file <none>
I20260924 00:16:57.927815 16807 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:57.927860 16807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:57.928004 16807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:57.929176 16807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "0a49182fc25e49e9b606b8931f3b3871"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d9b9a7cacc408eca4866e995ec7857f1"
server_key_iv: "274ed2e61e6a8634eeb28bda117b3213"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.929601 16807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "0a49182fc25e49e9b606b8931f3b3871"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d9b9a7cacc408eca4866e995ec7857f1"
server_key_iv: "274ed2e61e6a8634eeb28bda117b3213"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.930972 16807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:16:57.931705 16822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.931880 16807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.931968 16807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "0a49182fc25e49e9b606b8931f3b3871"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d9b9a7cacc408eca4866e995ec7857f1"
server_key_iv: "274ed2e61e6a8634eeb28bda117b3213"
server_key_version: "encryptionkey@0"
I20260924 00:16:57.932039 16807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:57.944602 16807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:57.946441 16807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:57.946578 16807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:57.946807 16807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:57.947098 16807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:57.947152 16807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.947183 16807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:57.947230 16807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:57.953171 16807 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:43629
I20260924 00:16:57.953220 16935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:43629 every 8 connection(s)
I20260924 00:16:57.953509 16807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:16:57.958904 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16807
I20260924 00:16:57.959016 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:16:57.959265 1202 external_mini_cluster.cc:1428] Setting key f3938de0e66aa4e0624cc3bfc6527ddb
Sep 24 00:16:57 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209017, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:57.963370 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:57.954175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:42045 (local address 127.1.44.190:35641)
0924 00:16:57.954248 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:57.954250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:57.954453 (+ 203us) server_negotiation.cc:407] Connection header received
0924 00:16:57.954922 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:57.954923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:57.954932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:57.954984 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:57.955983 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.956467 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.957387 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.957564 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.957613 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:57.960029 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:57.960031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:57.960031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:57.960039 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:57.961620 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:57.962235 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:57.962235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:57.962236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:57.962286 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:57.962633 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:57.962634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:57.962635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:57.962752 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:57.962817 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:57.963145 (+ 328us) server_negotiation.cc:299] Negotiation successful
0924 00:16:57.963285 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 00:16:57.963565 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:57.954168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:42045)
0924 00:16:57.954330 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:57.954370 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:57.954842 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:57.955044 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:57.955047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:57.955674 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:57.955897 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:57.955903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.956543 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.956544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:57.957313 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:57.957315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:57.957630 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:57.957631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:57.957724 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:57.958229 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:57.958237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:57.959929 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:57.961705 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:57.961711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:57.961713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:57.962153 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:57.962349 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:57.962351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:57.962351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:57.962515 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:57.962832 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:57.962838 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:57.963001 (+ 163us) client_negotiation.cc:769] Sending connection context
0924 00:16:57.963118 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 00:16:57.963314 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260924 00:16:57.964147 16936 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35641
I20260924 00:16:57.964252 16936 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:57.964445 16936 heartbeater.cc:511] Master 127.1.44.190:35641 requested a full tablet report, sending...
I20260924 00:16:57.964792 16611 ts_manager.cc:194] Registered new tserver with Master: 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629)
I20260924 00:16:57.965158 16611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:42045
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:16:57.985657 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35641
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:16:58.066272 16941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:16:58.066498 16941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:16:58.066537 16941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:16:58.067783 16941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:16:58.067844 16941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:16:58.067888 16941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:16:58.069324 16941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:16:58.069591 16941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:16:58.069821 16941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:16:58.074654 16949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:58.074666 16946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:16:58.074703 16947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:16:58.074922 16941 server_base.cc:1034] running on GCE node
I20260924 00:16:58.075142 16941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:16:58.075429 16941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:16:58.076601 16941 hybrid_clock.cc:648] HybridClock initialized: now 1790209018076574 us; error 41 us; skew 500 ppm
Sep 24 00:16:58 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209018, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:16:58.079083 16941 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:16:58.079550 16941 webserver.cc:460] Webserver started at http://127.1.44.131:36405/ using document root <none> and password file <none>
I20260924 00:16:58.079802 16941 fs_manager.cc:359] Metadata directory not provided
I20260924 00:16:58.079846 16941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:16:58.079932 16941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:16:58.080955 16941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "8818e2274d3e4d0c9d826cd56b9569eb"
format_stamp: "Formatted at 2026-09-24 00:16:58 on dist-test-slave-cm7b"
server_key: "8604479ac7c4c8f6d509e572fca80bf4"
server_key_iv: "eb85c32afc9dccff3ee7954808352d46"
server_key_version: "encryptionkey@0"
I20260924 00:16:58.081238 16941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "8818e2274d3e4d0c9d826cd56b9569eb"
format_stamp: "Formatted at 2026-09-24 00:16:58 on dist-test-slave-cm7b"
server_key: "8604479ac7c4c8f6d509e572fca80bf4"
server_key_iv: "eb85c32afc9dccff3ee7954808352d46"
server_key_version: "encryptionkey@0"
I20260924 00:16:58.082468 16941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:16:58.083108 16956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:58.083338 16941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:16:58.083406 16941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "8818e2274d3e4d0c9d826cd56b9569eb"
format_stamp: "Formatted at 2026-09-24 00:16:58 on dist-test-slave-cm7b"
server_key: "8604479ac7c4c8f6d509e572fca80bf4"
server_key_iv: "eb85c32afc9dccff3ee7954808352d46"
server_key_version: "encryptionkey@0"
I20260924 00:16:58.083455 16941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:16:58.089820 16941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:16:58.091459 16941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:16:58.091602 16941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:16:58.091802 16941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:16:58.092077 16941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:16:58.092111 16941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:58.092130 16941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:16:58.092175 16941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:16:58.097678 16941 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:41667
I20260924 00:16:58.097715 17069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:41667 every 8 connection(s)
I20260924 00:16:58.098037 16941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:16:58.099974 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 16941
I20260924 00:16:58.100100 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:16:58.100303 1202 external_mini_cluster.cc:1428] Setting key ac2e6db0edeee2dcff23cf58d68221de
Sep 24 00:16:58 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209018, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:16:58.107542 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.098738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44293 (local address 127.1.44.190:35641)
0924 00:16:58.098800 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.098802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.098972 (+ 170us) server_negotiation.cc:407] Connection header received
0924 00:16:58.099498 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.099499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.099509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.099563 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:58.100798 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.101410 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.102400 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.102612 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.102696 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.104692 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:58.104693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:58.104693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:58.104701 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:58.106042 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:58.106492 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:58.106493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:58.106494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:58.106558 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:58.106815 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:58.106816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:58.106816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:58.106939 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:58.107016 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.107365 (+ 349us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.107454 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 00:16:58.107748 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.098712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:44293)
0924 00:16:58.098854 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.098893 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:58.099415 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:58.099636 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:58.099638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:58.100403 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:16:58.100719 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.100725 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.101503 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.101505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.102296 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.102298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.102704 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.102705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.102826 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.103289 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:16:58.103298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:16:58.104597 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:16:58.106167 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:58.106174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:58.106176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:58.106398 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:58.106645 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:16:58.106646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:16:58.106646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:16:58.106746 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:16:58.107007 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:16:58.107013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:16:58.107150 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 00:16:58.107342 (+ 192us) client_negotiation.cc:240] Negotiation successful
0924 00:16:58.107497 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260924 00:16:58.108254 17070 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35641
I20260924 00:16:58.108364 17070 heartbeater.cc:464] Registering TS with master...
I20260924 00:16:58.108534 17070 heartbeater.cc:511] Master 127.1.44.190:35641 requested a full tablet report, sending...
I20260924 00:16:58.108970 16611 ts_manager.cc:194] Registered new tserver with Master: 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
I20260924 00:16:58.109298 16611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:44293
I20260924 00:16:58.112119 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:16:58.118961 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.113473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.1.44.190:35641)
0924 00:16:58.113517 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.113519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.113658 (+ 139us) server_negotiation.cc:407] Connection header received
0924 00:16:58.113688 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.113689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.113698 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.113736 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:16:58.114259 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.114712 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.115466 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.115641 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.115735 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.116752 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:16:58.116753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:16:58.116753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:16:58.116759 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:16:58.118079 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:58.118361 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:58.118361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:58.118362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:58.118408 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:16:58.118548 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:16:58.118548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:16:58.118548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:16:58.118668 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:16:58.118736 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.118826 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.118876 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:58.120713 16611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:16:58.122220 16611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:16:58.128895 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.124503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:40999 (local address 127.0.0.1:55718)
0924 00:16:58.124908 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.124917 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:58.124956 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:58.125386 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:58.125387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:58.125389 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:58.125420 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.125421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.126667 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.126668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.127751 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.127754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.128614 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.128614 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.128672 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.128674 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:16:58.128751 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 00:16:58.128809 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":29,"thread_start_us":61,"threads_started":1}
I20260924 00:16:58.128906 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.124727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.1.44.129:40999)
0924 00:16:58.125153 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.125159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.125167 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:16:58.125217 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.125219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.125236 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.125308 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:58.125507 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.126571 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.127851 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.128542 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.128605 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.128653 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.128768 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.128823 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":98,"threads_started":1}
I20260924 00:16:58.128964 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.124380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.1.44.131:41667)
0924 00:16:58.124519 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.124524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.124797 (+ 273us) server_negotiation.cc:407] Connection header received
0924 00:16:58.124901 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.124904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.124923 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.125026 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:58.125268 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.126042 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.127603 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.128201 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.128291 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.128350 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.128835 (+ 485us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.128879 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260924 00:16:58.128983 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.124479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.1.44.130:43629)
0924 00:16:58.124659 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.124663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.124878 (+ 215us) server_negotiation.cc:407] Connection header received
0924 00:16:58.124921 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.124923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.124937 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.125049 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:58.125268 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.126396 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.127756 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.128480 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.128562 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.128621 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.128857 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.128900 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260924 00:16:58.129247 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.124299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41667 (local address 127.0.0.1:41678)
0924 00:16:58.124661 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.124710 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:58.124800 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:58.125028 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:58.125098 (+ 70us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:58.125101 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:58.125173 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.125176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.126212 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.126212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.127531 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.127534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.128659 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.128660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.128732 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.128736 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:16:58.129060 (+ 324us) client_negotiation.cc:240] Negotiation successful
0924 00:16:58.129145 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":4,"thread_start_us":68,"threads_started":1}
I20260924 00:16:58.129240 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.124332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43629 (local address 127.0.0.1:36476)
0924 00:16:58.124713 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.124724 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:58.124800 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:58.125129 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:58.125130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:58.125132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:58.125186 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.125188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.126493 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.126494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.127671 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.127678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.128560 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.128561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.128655 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.128670 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 00:16:58.128744 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 00:16:58.128810 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":4,"thread_start_us":53,"threads_started":1}
I20260924 00:16:58.129989 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 (DEFAULT_TABLE table=test-table [id=2179d3c680014dfab6c53b379709ad29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:58.129987 17004 tablet_service.cc:1467] Processing CreateTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 (DEFAULT_TABLE table=test-table [id=2179d3c680014dfab6c53b379709ad29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:58.130306 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 665b04684835474ca2fc61fa8e6f4b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:58.130316 17004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 665b04684835474ca2fc61fa8e6f4b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:58.131814 16736 tablet_service.cc:1467] Processing CreateTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 (DEFAULT_TABLE table=test-table [id=2179d3c680014dfab6c53b379709ad29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:16:58.132133 16736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 665b04684835474ca2fc61fa8e6f4b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:16:58.133150 17084 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Bootstrap starting.
I20260924 00:16:58.133477 17085 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Bootstrap starting.
I20260924 00:16:58.133942 17084 tablet_bootstrap.cc:654] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:58.134119 17085 tablet_bootstrap.cc:654] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:58.134698 17084 log.cc:824] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:58.134724 17085 log.cc:824] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:58.135538 17088 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap starting.
I20260924 00:16:58.136175 17084 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: No bootstrap required, opened a new log
I20260924 00:16:58.136188 17085 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: No bootstrap required, opened a new log
I20260924 00:16:58.136309 17084 ts_tablet_manager.cc:1397] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:16:58.136384 17085 ts_tablet_manager.cc:1397] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:16:58.136407 17088 tablet_bootstrap.cc:654] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Neither blocks nor log segments found. Creating new log.
I20260924 00:16:58.137174 17088 log.cc:824] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:16:58.137912 17088 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: No bootstrap required, opened a new log
I20260924 00:16:58.137998 17088 ts_tablet_manager.cc:1397] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:16:58.138370 17085 raft_consensus.cc:348] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.138417 17084 raft_consensus.cc:348] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.138556 17084 raft_consensus.cc:374] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:58.138556 17085 raft_consensus.cc:374] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:58.138592 17085 raft_consensus.cc:729] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8818e2274d3e4d0c9d826cd56b9569eb, State: Initialized, Role: FOLLOWER
I20260924 00:16:58.138592 17084 raft_consensus.cc:729] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a49182fc25e49e9b606b8931f3b3871, State: Initialized, Role: FOLLOWER
I20260924 00:16:58.138718 17085 consensus_queue.cc:260] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.138723 17084 consensus_queue.cc:260] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.139000 17085 ts_tablet_manager.cc:1428] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:58.139000 17084 ts_tablet_manager.cc:1428] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 00:16:58.139029 17070 heartbeater.cc:503] Master 127.1.44.190:35641 was elected leader, sending a full tablet report...
I20260924 00:16:58.139117 16936 heartbeater.cc:503] Master 127.1.44.190:35641 was elected leader, sending a full tablet report...
I20260924 00:16:58.140260 17088 raft_consensus.cc:348] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.140456 17088 raft_consensus.cc:374] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:16:58.140506 17088 raft_consensus.cc:729] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51e1904fd0d44c9bf5b5e5093f942bc, State: Initialized, Role: FOLLOWER
I20260924 00:16:58.140614 17088 consensus_queue.cc:260] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.140875 17088 ts_tablet_manager.cc:1428] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:16:58.140935 16802 heartbeater.cc:503] Master 127.1.44.190:35641 was elected leader, sending a full tablet report...
W20260924 00:16:58.204047 16937 tablet.cc:2369] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:16:58.230134 17092 raft_consensus.cc:482] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:16:58.230258 17092 raft_consensus.cc:504] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.230567 17092 leader_election.cc:288] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667), 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629)
I20260924 00:16:58.233657 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.230828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43629 (local address 127.1.44.129:59609)
0924 00:16:58.230873 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.230886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:58.230933 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:58.231166 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:58.231167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:58.231169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:58.231280 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.231283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.231965 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.231965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.232798 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.232800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.233392 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.233393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.233479 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.233484 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:16:58.233539 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:16:58.233579 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260924 00:16:58.233690 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.230960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:59609 (local address 127.1.44.130:43629)
0924 00:16:58.231006 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.231008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.231018 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:16:58.231049 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.231050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.231063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.231138 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:58.231357 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.231880 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.232890 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.233311 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.233366 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.233373 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.233554 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.233595 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:16:58.233812 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.230892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:37689 (local address 127.1.44.131:41667)
0924 00:16:58.230941 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.230944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.231155 (+ 211us) server_negotiation.cc:407] Connection header received
0924 00:16:58.231223 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.231224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.231235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.231292 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:16:58.231483 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.232047 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.233017 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.233419 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.233490 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.233499 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.233678 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.233722 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:16:58.233970 17093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.230828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41667 (local address 127.1.44.129:37689)
0924 00:16:58.231052 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.231063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:16:58.231141 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:16:58.231302 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:16:58.231303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:16:58.231304 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:16:58.231404 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.231406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.232137 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.232138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.232916 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:16:58.232918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.233500 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.233501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:16:58.233586 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.233591 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:16:58.233649 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 00:16:58.233685 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":54,"thread_start_us":96,"threads_started":1}
I20260924 00:16:58.234087 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665b04684835474ca2fc61fa8e6f4b68" candidate_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a49182fc25e49e9b606b8931f3b3871" is_pre_election: true
I20260924 00:16:58.234261 16890 raft_consensus.cc:2454] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51e1904fd0d44c9bf5b5e5093f942bc in term 0.
I20260924 00:16:58.234323 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665b04684835474ca2fc61fa8e6f4b68" candidate_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" is_pre_election: true
I20260924 00:16:58.234483 17024 raft_consensus.cc:2454] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51e1904fd0d44c9bf5b5e5093f942bc in term 0.
I20260924 00:16:58.234512 16689 leader_election.cc:302] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a49182fc25e49e9b606b8931f3b3871, d51e1904fd0d44c9bf5b5e5093f942bc; no voters:
I20260924 00:16:58.234676 17092 raft_consensus.cc:2781] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:16:58.234745 17092 raft_consensus.cc:482] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:16:58.234763 17092 raft_consensus.cc:3037] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:58.235641 17092 raft_consensus.cc:504] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.235790 17092 leader_election.cc:288] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 1 election: Requested vote from peers 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667), 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629)
I20260924 00:16:58.235935 16890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665b04684835474ca2fc61fa8e6f4b68" candidate_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a49182fc25e49e9b606b8931f3b3871"
I20260924 00:16:58.235965 17024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "665b04684835474ca2fc61fa8e6f4b68" candidate_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8818e2274d3e4d0c9d826cd56b9569eb"
I20260924 00:16:58.236007 16890 raft_consensus.cc:3037] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:58.236052 17024 raft_consensus.cc:3037] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:16:58.236841 16890 raft_consensus.cc:2454] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51e1904fd0d44c9bf5b5e5093f942bc in term 1.
I20260924 00:16:58.236872 17024 raft_consensus.cc:2454] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51e1904fd0d44c9bf5b5e5093f942bc in term 1.
I20260924 00:16:58.237012 16689 leader_election.cc:302] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a49182fc25e49e9b606b8931f3b3871, d51e1904fd0d44c9bf5b5e5093f942bc; no voters:
I20260924 00:16:58.237098 17092 raft_consensus.cc:2781] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:16:58.237275 17092 raft_consensus.cc:686] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 LEADER]: Becoming Leader. State: Replica: d51e1904fd0d44c9bf5b5e5093f942bc, State: Running, Role: LEADER
I20260924 00:16:58.237378 17092 consensus_queue.cc:237] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:16:58.238185 16609 catalog_manager.cc:5495] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129). New cstate: current_term: 1 leader_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } health_report { overall_health: UNKNOWN } } }
I20260924 00:16:58.243137 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.240885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.1.44.129:40999)
0924 00:16:58.240924 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:16:58.240926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:16:58.240934 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:16:58.241005 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:16:58.241006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:16:58.241024 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:16:58.241086 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:16:58.241206 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.241642 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.242379 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:16:58.242582 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:16:58.242656 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:16:58.242815 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:16:58.242913 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:16:58.243003 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 00:16:58.243028 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 00:16:58.243045 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:16:58.244731 16890 raft_consensus.cc:1261] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Refusing update from remote peer d51e1904fd0d44c9bf5b5e5093f942bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:58.244731 17024 raft_consensus.cc:1261] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Refusing update from remote peer d51e1904fd0d44c9bf5b5e5093f942bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:16:58.245091 17094 consensus_queue.cc:1035] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:58.245198 17092 consensus_queue.cc:1035] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:16:58.247265 17097 mvcc.cc:205] Tried to move back new op lower bound from 7332696138727813120 to 7332696138700902400. Current Snapshot: MvccSnapshot[applied={T|T < 7332696138727813120}]
I20260924 00:16:58.247776 17099 mvcc.cc:205] Tried to move back new op lower bound from 7332696138727813120 to 7332696138700902400. Current Snapshot: MvccSnapshot[applied={T|T < 7332696138727813120}]
I20260924 00:16:58.249807 16611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:58.249898 16611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:16:58.250617 16611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Sending DeleteTablet for 3 replicas of tablet 665b04684835474ca2fc61fa8e6f4b68
I20260924 00:16:58.250976 16736 tablet_service.cc:1514] Processing DeleteTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:58 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:55718
I20260924 00:16:58.250957 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:58 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:36476
I20260924 00:16:58.251017 17004 tablet_service.cc:1514] Processing DeleteTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:58 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:41678
I20260924 00:16:58.251165 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16673
I20260924 00:16:58.251749 17108 tablet_replica.cc:330] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: stopping tablet replica
I20260924 00:16:58.251852 17108 raft_consensus.cc:2229] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:58.251946 17108 raft_consensus.cc:2258] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:58.252400 17108 ts_tablet_manager.cc:1905] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:58.253036 17109 tablet_replica.cc:330] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: stopping tablet replica
I20260924 00:16:58.253137 17109 raft_consensus.cc:2229] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:16:58.253240 17109 raft_consensus.cc:2258] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:16:58.253681 17109 ts_tablet_manager.cc:1905] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:16:58.254356 17108 ts_tablet_manager.cc:1918] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:58.254477 17108 log.cc:1196] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/665b04684835474ca2fc61fa8e6f4b68
I20260924 00:16:58.254829 17108 ts_tablet_manager.cc:1939] T 665b04684835474ca2fc61fa8e6f4b68 P 0a49182fc25e49e9b606b8931f3b3871: Deleting consensus metadata
I20260924 00:16:58.255304 16595 catalog_manager.cc:4841] TS 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): tablet 665b04684835474ca2fc61fa8e6f4b68 (table test-table [id=2179d3c680014dfab6c53b379709ad29]) successfully deleted
I20260924 00:16:58.255329 17109 ts_tablet_manager.cc:1918] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:16:58.255388 17109 log.cc:1196] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/665b04684835474ca2fc61fa8e6f4b68
I20260924 00:16:58.255640 17109 ts_tablet_manager.cc:1939] T 665b04684835474ca2fc61fa8e6f4b68 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting consensus metadata
W20260924 00:16:58.256036 16597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:40999 (error 108)
I20260924 00:16:58.256243 16597 catalog_manager.cc:4841] TS 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667): tablet 665b04684835474ca2fc61fa8e6f4b68 (table test-table [id=2179d3c680014dfab6c53b379709ad29]) successfully deleted
I20260924 00:16:58.256309 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16807
I20260924 00:16:58.256497 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:16:58.256393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:40999 (local address 127.0.0.1:55730)
0924 00:16:58.256435 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 00:16:58.256465 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 00:16:58.257273 16597 catalog_manager.cc:4584] TS d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 665b04684835474ca2fc61fa8e6f4b68: Network error: Client connection negotiation failed: client connection to 127.1.44.129:40999: connect: Connection refused (error 111)
I20260924 00:16:58.261277 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16941
I20260924 00:16:58.265640 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 16579
I20260924 00:17:18.270874 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35641
--webserver_interface=127.1.44.190
--webserver_port=38109
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:18.350451 17111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:18.350642 17111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:18.350663 17111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:18.351862 17111 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 00:17:18.351902 17111 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 00:17:18.351919 17111 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 00:17:18.351930 17111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:17:18.351943 17111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:18.351956 17111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:18.351969 17111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:17:18.351981 17111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:17:18.353451 17111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:35641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=38109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:18.353695 17111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:18.353926 17111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:17:18.362341 17111 server_base.cc:1034] running on GCE node
W20260924 00:17:18.362371 17117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:18.362494 17120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:18.362319 17118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:18.362916 17111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:18.363309 17111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:18.364476 17111 hybrid_clock.cc:648] HybridClock initialized: now 1790209038364463 us; error 28 us; skew 500 ppm
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:18.367041 17111 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:17:18.367517 17111 webserver.cc:460] Webserver started at http://127.1.44.190:38109/ using document root <none> and password file <none>
I20260924 00:17:18.367756 17111 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:18.367846 17111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:18.369144 17111 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:18.369835 17127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.370020 17111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:18.370128 17111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "d0c9f267f7e04239ae529d1bbf20fb92"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "9afe93a038dfe4b9abcb6a2a6432fd52"
server_key_iv: "60693015b8bae3626271a5b6a1e16c8f"
server_key_version: "encryptionkey@0"
I20260924 00:17:18.370491 17111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:18.382849 17111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:18.385095 17111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:18.385255 17111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:18.389288 17111 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:35641
I20260924 00:17:18.389315 17179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:35641 every 8 connection(s)
I20260924 00:17:18.389678 17111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:17:18.391134 17180 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:17:18.391798 17180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap starting.
I20260924 00:17:18.395674 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17111
I20260924 00:17:18.395942 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:40999
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:35641
--webserver_port=44129
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:17:18.400480 17180 log.cc:824] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:18.402225 17180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:17:18.402514 17180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap complete.
I20260924 00:17:18.404675 17180 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:18.404876 17180 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c9f267f7e04239ae529d1bbf20fb92, State: Initialized, Role: FOLLOWER
I20260924 00:17:18.405033 17180 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:18.405092 17180 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:17:18.405157 17180 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:17:18.405208 17180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 1 FOLLOWER]: Advancing to term 2
I20260924 00:17:18.406154 17180 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:18.406261 17180 leader_election.cc:302] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c9f267f7e04239ae529d1bbf20fb92; no voters:
I20260924 00:17:18.406457 17180 leader_election.cc:288] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 00:17:18.406488 17184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 00:17:18.406595 17184 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 LEADER]: Becoming Leader. State: Replica: d0c9f267f7e04239ae529d1bbf20fb92, State: Running, Role: LEADER
I20260924 00:17:18.406677 17184 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:18.406708 17180 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:17:18.407075 17186 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c9f267f7e04239ae529d1bbf20fb92. Latest consensus state: current_term: 2 leader_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } } }
I20260924 00:17:18.407155 17186 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:18.407405 17189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:17:18.408130 17189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:17:18.408246 17189 catalog_manager.cc:1259] Loaded cluster ID: 71abef1b8fb04329a9a5bc57803e7de3
I20260924 00:17:18.408447 17189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:17:18.408169 17185 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } } }
I20260924 00:17:18.408493 17185 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:18.409226 17189 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:17:18.409422 17189 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Loaded TSK: 0
I20260924 00:17:18.420581 17189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Generated new TSK 1
I20260924 00:17:18.420662 17189 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:17:18.477169 17182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:18.477375 17182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:18.477412 17182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:18.479004 17182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:18.479071 17182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:18.479115 17182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:17:18.480597 17182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:40999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=44129
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:18.480863 17182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:18.481076 17182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:18.486799 17210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:18.486914 17207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:18.486783 17208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:18.486999 17182 server_base.cc:1034] running on GCE node
I20260924 00:17:18.487224 17182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:18.487485 17182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:18.488668 17182 hybrid_clock.cc:648] HybridClock initialized: now 1790209038488655 us; error 27 us; skew 500 ppm
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:18.491503 17182 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:17:18.491920 17182 webserver.cc:460] Webserver started at http://127.1.44.129:44129/ using document root <none> and password file <none>
I20260924 00:17:18.492136 17182 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:18.492213 17182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:18.493563 17182 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:18.494269 17217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.494481 17182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:18.494563 17182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "d51e1904fd0d44c9bf5b5e5093f942bc"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d95f6564f6bc9adcde086f7f0e72d070"
server_key_iv: "97f776c05ed0013f647ec5e47c84fd99"
server_key_version: "encryptionkey@0"
I20260924 00:17:18.494894 17182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:18.506224 17182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:18.508496 17182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:18.508656 17182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:18.508885 17182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:18.509379 17224 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 00:17:18.511190 17182 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 00:17:18.511305 17182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 00:17:18.511346 17182 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 00:17:18.511909 17182 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 00:17:18.511965 17182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 00:17:18.512030 17224 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap starting.
I20260924 00:17:18.519027 17182 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:40999
I20260924 00:17:18.519138 17331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:40999 every 8 connection(s)
I20260924 00:17:18.519438 17182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:17:18.520752 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17182
I20260924 00:17:18.521052 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:43629
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:35641
--webserver_port=45853
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:17:18.522619 17224 log.cc:824] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:18.525194 17224 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:17:18.525490 17224 tablet_bootstrap.cc:492] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap complete.
I20260924 00:17:18.526680 17224 ts_tablet_manager.cc:1397] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.012s
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:18.527915 17224 raft_consensus.cc:348] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:17:18.528236 17224 raft_consensus.cc:729] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51e1904fd0d44c9bf5b5e5093f942bc, State: Initialized, Role: FOLLOWER
I20260924 00:17:18.531785 17334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.520202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:36151)
0924 00:17:18.520537 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.520615 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.521643 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.522232 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.522236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.522601 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:18.522867 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.522874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.524602 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.524603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.525465 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.525468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.526113 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.526114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.526248 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.526817 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:18.526827 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:18.528223 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:18.530198 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:18.530205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:18.530207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:18.530436 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:18.530687 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:18.530688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:18.530688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:18.530803 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:18.531156 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:18.531162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:18.531326 (+ 164us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.531435 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.531599 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":88,"threads_started":1}
I20260924 00:17:18.531809 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.520275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:36151 (local address 127.1.44.190:35641)
0924 00:17:18.520456 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.520466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.520719 (+ 253us) server_negotiation.cc:407] Connection header received
0924 00:17:18.521887 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.521896 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.521951 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.522053 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:18.523004 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.524485 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.525588 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.526002 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.526134 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.528356 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:18.528360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:18.528360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:18.528373 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:18.530099 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.530521 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.530524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.530524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.530613 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.530874 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.530875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.530875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.531080 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:18.531173 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.531487 (+ 314us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.531601 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260924 00:17:18.532800 17332 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35641
I20260924 00:17:18.532888 17332 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:18.533049 17332 heartbeater.cc:511] Master 127.1.44.190:35641 requested a full tablet report, sending...
I20260924 00:17:18.535331 17224 consensus_queue.cc:260] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } } peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } }
I20260924 00:17:18.536048 17224 ts_tablet_manager.cc:1428] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent starting tablet: real 0.009s user 0.003s sys 0.002s
I20260924 00:17:18.536280 17144 ts_manager.cc:194] Registered new tserver with Master: d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999)
I20260924 00:17:18.537251 17144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:36151
I20260924 00:17:18.538177 17332 heartbeater.cc:503] Master 127.1.44.190:35641 was elected leader, sending a full tablet report...
I20260924 00:17:18.541306 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.536981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.1.44.129:40999)
0924 00:17:18.537836 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.537840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.537849 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:17:18.537877 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.537878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.538022 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.538104 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:18.538391 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.539363 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.540460 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.540897 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.540978 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.541029 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.541208 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.541243 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":57,"threads_started":1}
I20260924 00:17:18.541319 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.536761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:40999 (local address 127.0.0.1:37696)
0924 00:17:18.537573 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.537605 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.537661 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.538160 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.538162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.538164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:18.538311 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.538314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.539444 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.539446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.540363 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.540366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.540991 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.540992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.541082 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.541094 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.541192 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.541243 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":85,"threads_started":1}
I20260924 00:17:18.541911 17266 tablet_service.cc:1514] Processing DeleteTablet for tablet 665b04684835474ca2fc61fa8e6f4b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:16:58 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:37696
I20260924 00:17:18.542362 17341 tablet_replica.cc:330] stopping tablet replica
I20260924 00:17:18.542477 17341 raft_consensus.cc:2229] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:18.542538 17341 raft_consensus.cc:2258] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:18.542806 17341 ts_tablet_manager.cc:1905] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:18.545097 17341 ts_tablet_manager.cc:1918] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:18.545169 17341 log.cc:1196] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal/wals/665b04684835474ca2fc61fa8e6f4b68
I20260924 00:17:18.545403 17341 ts_tablet_manager.cc:1939] T 665b04684835474ca2fc61fa8e6f4b68 P d51e1904fd0d44c9bf5b5e5093f942bc: Deleting consensus metadata
I20260924 00:17:18.545898 17130 catalog_manager.cc:4841] TS d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999): tablet 665b04684835474ca2fc61fa8e6f4b68 (table test-table [id=2179d3c680014dfab6c53b379709ad29]) successfully deleted
W20260924 00:17:18.612900 17336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:18.613077 17336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:18.613101 17336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:18.614272 17336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:18.614311 17336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:18.614336 17336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:17:18.615697 17336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:43629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=45853
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:18.615856 17336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:18.616014 17336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:18.620656 17346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:18.620645 17349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:18.620690 17336 server_base.cc:1034] running on GCE node
W20260924 00:17:18.620646 17347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:18.621023 17336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:18.621268 17336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:18.622411 17336 hybrid_clock.cc:648] HybridClock initialized: now 1790209038622391 us; error 35 us; skew 500 ppm
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:18.624714 17336 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:17:18.625099 17336 webserver.cc:460] Webserver started at http://127.1.44.130:45853/ using document root <none> and password file <none>
I20260924 00:17:18.625303 17336 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:18.625355 17336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:18.626605 17336 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:18.627146 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.627334 17336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.627396 17336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "0a49182fc25e49e9b606b8931f3b3871"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d9b9a7cacc408eca4866e995ec7857f1"
server_key_iv: "274ed2e61e6a8634eeb28bda117b3213"
server_key_version: "encryptionkey@0"
I20260924 00:17:18.627660 17336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:18.647495 17336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:18.649636 17336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:18.649739 17336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:18.649910 17336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:18.650126 17336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:18.650154 17336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.650174 17336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:18.650187 17336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.655735 17336 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:43629
I20260924 00:17:18.655782 17469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:43629 every 8 connection(s)
I20260924 00:17:18.656091 17336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:17:18.661458 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17336
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:18.661723 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:41667
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:35641
--webserver_port=36405
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:17:18.664928 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.656740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:36143 (local address 127.1.44.190:35641)
0924 00:17:18.656803 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.656806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.657024 (+ 218us) server_negotiation.cc:407] Connection header received
0924 00:17:18.657433 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.657434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.657447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.657497 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:18.658342 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.658814 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.659776 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.659965 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.660016 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.662051 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:18.662053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:18.662053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:18.662061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:18.663559 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.664049 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.664049 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.664050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.664095 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.664357 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.664357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.664357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.664448 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:18.664495 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.664796 (+ 301us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.664859 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 00:17:18.665207 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.656734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:36143)
0924 00:17:18.656911 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.656943 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.657356 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.657544 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.657547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.658098 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:18.658272 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.658276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.658888 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.658889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.659708 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.659710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.660036 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.660036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.660125 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.660589 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:18.660597 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:18.661968 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:18.663637 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:18.663643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:18.663645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:18.663965 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:18.664161 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:18.664162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:18.664162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:18.664293 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:18.664510 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:18.664515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:18.664668 (+ 153us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.664778 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.664946 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260924 00:17:18.665699 17470 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35641
I20260924 00:17:18.665802 17470 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:18.665961 17470 heartbeater.cc:511] Master 127.1.44.190:35641 requested a full tablet report, sending...
I20260924 00:17:18.666339 17144 ts_manager.cc:194] Registered new tserver with Master: 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629)
I20260924 00:17:18.666857 17144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:36143
W20260924 00:17:18.741477 17473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:18.741662 17473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:18.741684 17473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:18.742743 17473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:18.742774 17473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:18.742800 17473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:17:18.744207 17473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:41667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=36405
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:18.744400 17473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:18.744562 17473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:18.749552 17481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:18.749557 17473 server_base.cc:1034] running on GCE node
W20260924 00:17:18.749523 17478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:18.749516 17479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:18.749894 17473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:18.750135 17473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:18.751278 17473 hybrid_clock.cc:648] HybridClock initialized: now 1790209038751272 us; error 39 us; skew 500 ppm
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:18.753866 17473 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:17:18.754343 17473 webserver.cc:460] Webserver started at http://127.1.44.131:36405/ using document root <none> and password file <none>
I20260924 00:17:18.754580 17473 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:18.754662 17473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:18.756039 17473 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:18.756654 17488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.756821 17473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:17:18.756919 17473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "8818e2274d3e4d0c9d826cd56b9569eb"
format_stamp: "Formatted at 2026-09-24 00:16:58 on dist-test-slave-cm7b"
server_key: "8604479ac7c4c8f6d509e572fca80bf4"
server_key_iv: "eb85c32afc9dccff3ee7954808352d46"
server_key_version: "encryptionkey@0"
I20260924 00:17:18.757251 17473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:18.766286 17473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:18.768420 17473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:18.768569 17473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:18.768787 17473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:18.769057 17473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:18.769107 17473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.769145 17473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:18.769172 17473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.775280 17473 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:41667
I20260924 00:17:18.775341 17601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:41667 every 8 connection(s)
I20260924 00:17:18.775625 17473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:17:18.776995 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17473
Sep 24 00:17:18 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:18.785295 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.777480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.1.44.190:35641)
0924 00:17:18.777599 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.777602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.777626 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:17:18.777684 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.777685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.777696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.777759 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:18.778383 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.778908 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.779712 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.779970 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.780018 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.781187 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:18.781189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:18.781189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:18.781196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:18.784149 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.784493 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.784494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.784495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.784549 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.784727 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.784728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.784728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.784869 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:18.785056 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.785105 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.785185 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260924 00:17:18.785764 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.776297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44087 (local address 127.1.44.190:35641)
0924 00:17:18.776374 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.776377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.776528 (+ 151us) server_negotiation.cc:407] Connection header received
0924 00:17:18.776982 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.776983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.776995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.777072 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:18.778177 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.778750 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.779794 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.779984 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.780070 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.782455 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:18.782456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:18.782456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:18.782467 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:18.784304 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.784796 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.784797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.784797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.784840 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:18.785117 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:18.785117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:18.785118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:18.785226 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:18.785276 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.785628 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.785706 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":42}
I20260924 00:17:18.786198 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.776267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:44087)
0924 00:17:18.776419 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.776449 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.776883 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.777149 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.777153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.777910 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:18.778087 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.778091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.778838 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.778839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.779712 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.779715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.780096 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.780097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.780191 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.780704 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:18.780711 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:18.782346 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:18.784375 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:18.784381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:18.784383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:18.784726 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:18.784904 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:18.784904 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:18.784905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:18.785028 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:18.785325 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:18.785331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:18.785482 (+ 151us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.785833 (+ 351us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.786020 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
I20260924 00:17:18.786641 17602 heartbeater.cc:347] Connected to a master server at 127.1.44.190:35641
I20260924 00:17:18.786741 17602 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:18.786907 17602 heartbeater.cc:511] Master 127.1.44.190:35641 requested a full tablet report, sending...
I20260924 00:17:18.787292 17144 ts_manager.cc:194] Registered new tserver with Master: 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
I20260924 00:17:18.787591 17144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:44087
I20260924 00:17:18.796828 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:17:18.799496 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.797146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.1.44.190:35641)
0924 00:17:18.797208 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.797210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.797316 (+ 106us) server_negotiation.cc:407] Connection header received
0924 00:17:18.797385 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.797385 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.797412 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.797465 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:17:18.797573 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.798037 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.798801 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.798973 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.799024 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.799144 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:17:18.799278 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:17:18.799338 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.799404 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.799418 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 00:17:18.799784 17144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:17:18.801306 17144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:17:18.803648 17266 tablet_service.cc:1467] Processing CreateTablet for tablet c8ed489a049c4af595f424faff78d655 (DEFAULT_TABLE table=test-table [id=43a4bad4ae324f9d931125c33bf85a71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:18.803849 17266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ed489a049c4af595f424faff78d655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:18.805471 17224 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap starting.
I20260924 00:17:18.805783 17224 tablet_bootstrap.cc:654] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:18.806366 17224 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: No bootstrap required, opened a new log
I20260924 00:17:18.806430 17224 ts_tablet_manager.cc:1397] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 00:17:18.806540 17224 raft_consensus.cc:348] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.806625 17224 raft_consensus.cc:374] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:18.806675 17224 raft_consensus.cc:729] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51e1904fd0d44c9bf5b5e5093f942bc, State: Initialized, Role: FOLLOWER
I20260924 00:17:18.806721 17224 consensus_queue.cc:260] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.806820 17224 ts_tablet_manager.cc:1428] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:18.807134 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.803122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43629 (local address 127.0.0.1:58734)
0924 00:17:18.803167 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.803176 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.803246 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.803619 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.803620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.803622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:18.803660 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.803662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.804584 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.804585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.805382 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.805383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.806879 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.806880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.806947 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.806951 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.807008 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.807051 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 00:17:18.807384 17610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.803446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.1.44.131:41667)
0924 00:17:18.803617 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.803621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.803631 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:17:18.803665 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.803667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.803682 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.803772 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:18.804028 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.805057 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.806176 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.806867 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.806923 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.806968 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.807261 (+ 293us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.807310 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260924 00:17:18.807543 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.803181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41667 (local address 127.0.0.1:38394)
0924 00:17:18.803361 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.803370 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.803409 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.803839 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.803839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.803841 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:18.803871 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.803873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.805149 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.805150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.806090 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.806093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.806982 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.806983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.807075 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.807079 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.807129 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.807170 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":138,"threads_started":1}
I20260924 00:17:18.807565 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.803273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.1.44.130:43629)
0924 00:17:18.803461 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.803465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.803476 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:17:18.803521 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.803523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.803540 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.803622 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:18.803713 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.804498 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.806018 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.806762 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.806871 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.806935 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.807424 (+ 489us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.807485 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260924 00:17:18.808192 17404 tablet_service.cc:1467] Processing CreateTablet for tablet c8ed489a049c4af595f424faff78d655 (DEFAULT_TABLE table=test-table [id=43a4bad4ae324f9d931125c33bf85a71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:18.808393 17536 tablet_service.cc:1467] Processing CreateTablet for tablet c8ed489a049c4af595f424faff78d655 (DEFAULT_TABLE table=test-table [id=43a4bad4ae324f9d931125c33bf85a71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:18.808518 17404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ed489a049c4af595f424faff78d655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:18.808691 17536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ed489a049c4af595f424faff78d655. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:18.811224 17612 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Bootstrap starting.
I20260924 00:17:18.811340 17613 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Bootstrap starting.
I20260924 00:17:18.811831 17612 tablet_bootstrap.cc:654] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:18.811877 17613 tablet_bootstrap.cc:654] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:18.812418 17612 log.cc:824] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:18.812449 17613 log.cc:824] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:18.813139 17613 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: No bootstrap required, opened a new log
I20260924 00:17:18.813169 17612 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: No bootstrap required, opened a new log
I20260924 00:17:18.813201 17613 ts_tablet_manager.cc:1397] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:18.813235 17612 ts_tablet_manager.cc:1397] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:18.814486 17613 raft_consensus.cc:348] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.814637 17613 raft_consensus.cc:374] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:18.814566 17612 raft_consensus.cc:348] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.814682 17613 raft_consensus.cc:729] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8818e2274d3e4d0c9d826cd56b9569eb, State: Initialized, Role: FOLLOWER
I20260924 00:17:18.814723 17612 raft_consensus.cc:374] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:18.814777 17612 raft_consensus.cc:729] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a49182fc25e49e9b606b8931f3b3871, State: Initialized, Role: FOLLOWER
I20260924 00:17:18.814795 17613 consensus_queue.cc:260] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.814919 17612 consensus_queue.cc:260] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.815040 17602 heartbeater.cc:503] Master 127.1.44.190:35641 was elected leader, sending a full tablet report...
I20260924 00:17:18.815224 17612 ts_tablet_manager.cc:1428] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:18.815261 17470 heartbeater.cc:503] Master 127.1.44.190:35641 was elected leader, sending a full tablet report...
I20260924 00:17:18.815301 17613 ts_tablet_manager.cc:1428] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:18.900293 17616 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:18.900408 17616 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.900738 17616 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999)
I20260924 00:17:18.903406 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.900983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:55023 (local address 127.1.44.130:43629)
0924 00:17:18.901034 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.901036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.901047 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:17:18.901078 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.901079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.901089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.901155 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:18.901326 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.901796 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.902688 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.903062 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.903145 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.903154 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.903314 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.903351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 00:17:18.903406 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.900899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43629 (local address 127.1.44.131:55023)
0924 00:17:18.900946 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.900959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.901024 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.901173 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.901173 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.901175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:18.901242 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.901243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.901880 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.901881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.902595 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.902597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.903142 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.903143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.903238 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.903241 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.903296 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.903334 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 00:17:18.903574 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.900976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:45853 (local address 127.1.44.129:40999)
0924 00:17:18.901020 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.901022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.901152 (+ 130us) server_negotiation.cc:407] Connection header received
0924 00:17:18.901240 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.901241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.901258 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.901319 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:18.901466 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.901924 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.902853 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.903238 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.903328 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.903336 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.903473 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.903512 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:17:18.903589 17618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.900903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:40999 (local address 127.1.44.131:45853)
0924 00:17:18.901096 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.901106 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:18.901176 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:18.901324 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:18.901325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:18.901326 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:18.901397 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.901398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.902009 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.902010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.902761 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:18.902762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.903327 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.903328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:18.903401 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.903404 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:18.903469 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 00:17:18.903504 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":39,"thread_start_us":82,"threads_started":1}
I20260924 00:17:18.903860 17424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ed489a049c4af595f424faff78d655" candidate_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a49182fc25e49e9b606b8931f3b3871" is_pre_election: true
I20260924 00:17:18.903918 17285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ed489a049c4af595f424faff78d655" candidate_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" is_pre_election: true
I20260924 00:17:18.904038 17424 raft_consensus.cc:2454] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8818e2274d3e4d0c9d826cd56b9569eb in term 0.
I20260924 00:17:18.904062 17285 raft_consensus.cc:2454] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8818e2274d3e4d0c9d826cd56b9569eb in term 0.
I20260924 00:17:18.904251 17491 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8818e2274d3e4d0c9d826cd56b9569eb, d51e1904fd0d44c9bf5b5e5093f942bc; no voters:
I20260924 00:17:18.904399 17616 raft_consensus.cc:2781] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:17:18.904452 17616 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:17:18.904475 17616 raft_consensus.cc:3037] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:18.905154 17616 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.905287 17616 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [CANDIDATE]: Term 1 election: Requested vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999)
I20260924 00:17:18.905402 17285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ed489a049c4af595f424faff78d655" candidate_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc"
I20260924 00:17:18.905494 17285 raft_consensus.cc:3037] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:18.905514 17424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ed489a049c4af595f424faff78d655" candidate_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a49182fc25e49e9b606b8931f3b3871"
I20260924 00:17:18.905598 17424 raft_consensus.cc:3037] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:18.906356 17424 raft_consensus.cc:2454] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8818e2274d3e4d0c9d826cd56b9569eb in term 1.
I20260924 00:17:18.906497 17285 raft_consensus.cc:2454] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8818e2274d3e4d0c9d826cd56b9569eb in term 1.
W20260924 00:17:18.906582 17471 tablet.cc:2369] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:17:18.906658 17491 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8818e2274d3e4d0c9d826cd56b9569eb, d51e1904fd0d44c9bf5b5e5093f942bc; no voters:
I20260924 00:17:18.906759 17616 raft_consensus.cc:2781] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:18.906919 17616 raft_consensus.cc:686] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 LEADER]: Becoming Leader. State: Replica: 8818e2274d3e4d0c9d826cd56b9569eb, State: Running, Role: LEADER
I20260924 00:17:18.907012 17616 consensus_queue.cc:237] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:18.907908 17142 catalog_manager.cc:5495] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131). New cstate: current_term: 1 leader_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } health_report { overall_health: UNKNOWN } } }
I20260924 00:17:18.923347 17610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.921210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.1.44.131:41667)
0924 00:17:18.921262 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:18.921276 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:18.921283 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:17:18.921329 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:18.921329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:18.921342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:18.921401 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:17:18.921513 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.921925 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.922604 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:18.922789 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:18.922845 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:18.923007 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:17:18.923093 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:17:18.923163 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:18.923262 (+ 99us) server_negotiation.cc:299] Negotiation successful
0924 00:17:18.923287 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:17:18.924928 17285 raft_consensus.cc:1261] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Refusing update from remote peer 8818e2274d3e4d0c9d826cd56b9569eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:18.924929 17424 raft_consensus.cc:1261] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Refusing update from remote peer 8818e2274d3e4d0c9d826cd56b9569eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:18.925259 17619 consensus_queue.cc:1035] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:18.925367 17616 consensus_queue.cc:1035] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:18.927819 17623 mvcc.cc:205] Tried to move back new op lower bound from 7332696223434067968 to 7332696223364018176. Current Snapshot: MvccSnapshot[applied={T|T < 7332696223434067968}]
I20260924 00:17:18.928136 17624 mvcc.cc:205] Tried to move back new op lower bound from 7332696223434067968 to 7332696223364018176. Current Snapshot: MvccSnapshot[applied={T|T < 7332696223434067968}]
I20260924 00:17:18.929629 17143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:18.929718 17143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:18.930384 17143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Sending DeleteTablet for 3 replicas of tablet c8ed489a049c4af595f424faff78d655
I20260924 00:17:18.930684 17536 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ed489a049c4af595f424faff78d655 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:18 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:38394
I20260924 00:17:18.930713 17266 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ed489a049c4af595f424faff78d655 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:18 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:37696
I20260924 00:17:18.930792 17341 tablet_replica.cc:330] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: stopping tablet replica
I20260924 00:17:18.930775 17404 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ed489a049c4af595f424faff78d655 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:18 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:58734
I20260924 00:17:18.930848 17341 raft_consensus.cc:2229] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:18.930896 17341 raft_consensus.cc:2258] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:18.931010 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 17182
I20260924 00:17:18.931006 17633 tablet_replica.cc:330] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: stopping tablet replica
I20260924 00:17:18.931155 17633 raft_consensus.cc:2229] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:18.931183 17341 ts_tablet_manager.cc:1905] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:18.931370 17633 raft_consensus.cc:2258] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:18.931867 17633 ts_tablet_manager.cc:1905] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:18.933928 17633 ts_tablet_manager.cc:1918] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:18.934041 17633 log.cc:1196] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/c8ed489a049c4af595f424faff78d655
I20260924 00:17:18.934373 17633 ts_tablet_manager.cc:1939] T c8ed489a049c4af595f424faff78d655 P 0a49182fc25e49e9b606b8931f3b3871: Deleting consensus metadata
I20260924 00:17:18.934803 17128 catalog_manager.cc:4841] TS 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): tablet c8ed489a049c4af595f424faff78d655 (table test-table [id=43a4bad4ae324f9d931125c33bf85a71]) successfully deleted
W20260924 00:17:18.937475 17130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:40999 (error 108)
I20260924 00:17:18.937704 17632 tablet_replica.cc:330] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: stopping tablet replica
I20260924 00:17:18.937794 17632 raft_consensus.cc:2229] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:17:18.937880 17632 raft_consensus.cc:2258] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:18.937882 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.937775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:40999 (local address 127.0.0.1:37698)
0924 00:17:18.937821 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.937850 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 00:17:18.938050 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 17336
W20260924 00:17:18.937975 17130 catalog_manager.cc:4584] TS d51e1904fd0d44c9bf5b5e5093f942bc (127.1.44.129:40999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8ed489a049c4af595f424faff78d655: Network error: Client connection negotiation failed: client connection to 127.1.44.129:40999: connect: Connection refused (error 111)
I20260924 00:17:18.943562 17632 ts_tablet_manager.cc:1905] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:18.943989 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 17473
I20260924 00:17:18.948589 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 17111
I20260924 00:17:18.948623 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:18.948491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41667 (local address 127.0.0.1:38412)
0924 00:17:18.948560 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:18.948591 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.131:41667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 00:17:38.953964 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35641
--webserver_interface=127.1.44.190
--webserver_port=38109
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:39.032980 17635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:39.033238 17635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:39.033274 17635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:39.034457 17635 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 00:17:39.034529 17635 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 00:17:39.034554 17635 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 00:17:39.034572 17635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:17:39.034596 17635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:39.034628 17635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:39.034652 17635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:17:39.034683 17635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:17:39.036175 17635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:35641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=38109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:39.036459 17635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:39.036690 17635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:39.041528 17643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:39.041508 17641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:39.041615 17640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:39.041674 17635 server_base.cc:1034] running on GCE node
I20260924 00:17:39.041944 17635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:39.042227 17635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:39.043366 17635 hybrid_clock.cc:648] HybridClock initialized: now 1790209059043355 us; error 28 us; skew 500 ppm
Sep 24 00:17:39 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209059, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:39.045814 17635 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:17:39.046341 17635 webserver.cc:460] Webserver started at http://127.1.44.190:38109/ using document root <none> and password file <none>
I20260924 00:17:39.046610 17635 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:39.046657 17635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:39.048038 17635 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:39.048717 17650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.048923 17635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:39.048995 17635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "d0c9f267f7e04239ae529d1bbf20fb92"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "9afe93a038dfe4b9abcb6a2a6432fd52"
server_key_iv: "60693015b8bae3626271a5b6a1e16c8f"
server_key_version: "encryptionkey@0"
I20260924 00:17:39.049363 17635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:39.065001 17635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:39.067065 17635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:39.067240 17635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:39.071185 17635 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:35641
I20260924 00:17:39.071285 17702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:35641 every 8 connection(s)
I20260924 00:17:39.071691 17635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:17:39.073166 17703 sys_catalog.cc:263] Verifying existing consensus state
I20260924 00:17:39.073830 17703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap starting.
I20260924 00:17:39.078855 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17635
I20260924 00:17:39.079185 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:40999
--local_ip_for_outbound_sockets=127.1.44.129
--tserver_master_addrs=127.1.44.190:35641
--webserver_port=44129
--webserver_interface=127.1.44.129
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:17:39.082599 17703 log.cc:824] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:39.084858 17703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:17:39.085170 17703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Bootstrap complete.
I20260924 00:17:39.087428 17703 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:39.087644 17703 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c9f267f7e04239ae529d1bbf20fb92, State: Initialized, Role: FOLLOWER
I20260924 00:17:39.087813 17703 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:39.087872 17703 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:17:39.087937 17703 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:17:39.087988 17703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 2 FOLLOWER]: Advancing to term 3
I20260924 00:17:39.088979 17703 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:39.089063 17703 leader_election.cc:302] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c9f267f7e04239ae529d1bbf20fb92; no voters:
I20260924 00:17:39.089263 17703 leader_election.cc:288] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 00:17:39.089298 17707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 00:17:39.089474 17707 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [term 3 LEADER]: Becoming Leader. State: Replica: d0c9f267f7e04239ae529d1bbf20fb92, State: Running, Role: LEADER
I20260924 00:17:39.089541 17703 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:17:39.089656 17707 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } }
I20260924 00:17:39.090016 17707 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c9f267f7e04239ae529d1bbf20fb92. Latest consensus state: current_term: 3 leader_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } } }
I20260924 00:17:39.090024 17709 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c9f267f7e04239ae529d1bbf20fb92" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 35641 } } }
I20260924 00:17:39.090215 17709 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:39.090526 17716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:17:39.090637 17707 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:39.091321 17716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:17:39.091492 17716 catalog_manager.cc:1259] Loaded cluster ID: 71abef1b8fb04329a9a5bc57803e7de3
I20260924 00:17:39.091706 17716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:17:39.092917 17716 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:17:39.093093 17716 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Loaded TSK: 0
I20260924 00:17:39.093142 17716 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0c9f267f7e04239ae529d1bbf20fb92: Loaded TSK: 1
F20260924 00:17:39.093361 17716 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faa1cc339ad google::LogMessage::Fail() at ??:0
@ 0x7faa1cc3688a google::LogMessage::SendToLog() at ??:0
@ 0x7faa1cc334de google::LogMessage::Flush() at ??:0
@ 0x7faa1cc34a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faa1f3a0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7faa1f39dd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7faa24b911cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7faa24b9148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7faa24b91594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7faa24b57c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7faa24b61ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faa1eb010d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faa1eaf6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7faa1f95a6db start_thread at ??:0
@ 0x7faa1b73161f clone at ??:0
W20260924 00:17:39.191865 17705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:39.192057 17705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:39.192090 17705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:39.193279 17705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:39.193320 17705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:39.193346 17705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:17:39.194772 17705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:40999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=44129
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:39.195015 17705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:39.195314 17705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:39.200301 17745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:39.200296 17744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:39.200562 17705 server_base.cc:1034] running on GCE node
W20260924 00:17:39.200672 17747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:39.200855 17705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:39.201149 17705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:39.202311 17705 hybrid_clock.cc:648] HybridClock initialized: now 1790209059202298 us; error 27 us; skew 500 ppm
Sep 24 00:17:39 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209059, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:39.205430 17705 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:17:39.205945 17705 webserver.cc:460] Webserver started at http://127.1.44.129:44129/ using document root <none> and password file <none>
I20260924 00:17:39.206236 17705 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:39.206319 17705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:39.208045 17705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:39.208847 17754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.209041 17705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:39.209143 17705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "d51e1904fd0d44c9bf5b5e5093f942bc"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d95f6564f6bc9adcde086f7f0e72d070"
server_key_iv: "97f776c05ed0013f647ec5e47c84fd99"
server_key_version: "encryptionkey@0"
I20260924 00:17:39.209471 17705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:39.221640 17705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:39.223889 17705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:39.224023 17705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:39.224260 17705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:39.224694 17761 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 00:17:39.226738 17705 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 00:17:39.226820 17705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 00:17:39.226873 17705 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 00:17:39.227581 17705 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 00:17:39.227636 17705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:39.227710 17761 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap starting.
I20260924 00:17:39.229573 17761 log.cc:824] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:39.232481 17761 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 00:17:39.232780 17761 tablet_bootstrap.cc:492] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Bootstrap complete.
I20260924 00:17:39.232955 17761 ts_tablet_manager.cc:1397] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260924 00:17:39.234302 17705 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:40999
I20260924 00:17:39.234339 17869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:40999 every 8 connection(s)
I20260924 00:17:39.234658 17705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:17:39.234695 17761 raft_consensus.cc:348] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:39.235154 17761 raft_consensus.cc:729] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51e1904fd0d44c9bf5b5e5093f942bc, State: Initialized, Role: FOLLOWER
I20260924 00:17:39.235319 17761 consensus_queue.cc:260] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:39.235597 17761 ts_tablet_manager.cc:1428] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:17:39.241204 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17705
I20260924 00:17:39.241463 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:43629
--local_ip_for_outbound_sockets=127.1.44.130
--tserver_master_addrs=127.1.44.190:35641
--webserver_port=45853
--webserver_interface=127.1.44.130
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:39.336335 17873 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:17:39.236045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:40739)
0924 00:17:39.236211 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.236250 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:39.236381 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:39.336122 (+ 99741us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
W20260924 00:17:39.336591 17870 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:35641 (0 consecutive failures): Network error: Failed to ping master at 127.1.44.190:35641: Client connection negotiation failed: client connection to 127.1.44.190:35641: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:17:39.336975 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.336828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:45981)
0924 00:17:39.336904 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.336951 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260924 00:17:39.337366 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.337267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:56747)
0924 00:17:39.337316 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.337341 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 00:17:39.337471 17870 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 00:17:39.352424 17874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:39.352658 17874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:39.352697 17874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:39.353860 17874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:39.353930 17874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:39.353977 17874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:17:39.355499 17874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:43629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=45853
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:39.355753 17874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:39.355970 17874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:39.360898 17880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:39.361042 17874 server_base.cc:1034] running on GCE node
W20260924 00:17:39.360910 17882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:39.360956 17879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:39.361325 17874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:39.361579 17874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:39.362746 17874 hybrid_clock.cc:648] HybridClock initialized: now 1790209059362736 us; error 26 us; skew 500 ppm
Sep 24 00:17:39 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209059, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:39.365067 17874 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:17:39.365520 17874 webserver.cc:460] Webserver started at http://127.1.44.130:45853/ using document root <none> and password file <none>
I20260924 00:17:39.365741 17874 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:39.365808 17874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:39.367153 17874 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:39.367877 17889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.368050 17874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:17:39.368144 17874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "0a49182fc25e49e9b606b8931f3b3871"
format_stamp: "Formatted at 2026-09-24 00:16:57 on dist-test-slave-cm7b"
server_key: "d9b9a7cacc408eca4866e995ec7857f1"
server_key_iv: "274ed2e61e6a8634eeb28bda117b3213"
server_key_version: "encryptionkey@0"
I20260924 00:17:39.368479 17874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:39.385613 17874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:39.387691 17874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:39.387851 17874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:39.388078 17874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:39.388381 17874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:39.388433 17874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.388474 17874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:39.388501 17874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.394344 17874 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:43629
I20260924 00:17:39.394395 18002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:43629 every 8 connection(s)
I20260924 00:17:39.394714 17874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:17:39.395714 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.395366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:42531)
0924 00:17:39.395517 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.395585 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
W20260924 00:17:39.396106 18003 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:35641 (0 consecutive failures): Network error: Failed to ping master at 127.1.44.190:35641: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
I20260924 00:17:39.396384 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.396284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:36925)
0924 00:17:39.396333 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.396358 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 00:17:39.396659 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.396576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:35277)
0924 00:17:39.396619 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.396641 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 00:17:39.396763 18003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:17:39.397081 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 17874
I20260924 00:17:39.397347 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:41667
--local_ip_for_outbound_sockets=127.1.44.131
--tserver_master_addrs=127.1.44.190:35641
--webserver_port=36405
--webserver_interface=127.1.44.131
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:39.476471 18006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:39.476677 18006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:39.476701 18006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:39.477854 18006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:39.477895 18006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:39.477921 18006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:17:39.479377 18006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:41667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=36405
--tserver_master_addrs=127.1.44.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:39.479624 18006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:39.479843 18006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:39.484935 18014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:39.484956 18011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:39.484949 18012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:39.485157 18006 server_base.cc:1034] running on GCE node
I20260924 00:17:39.485390 18006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:39.485674 18006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:39.486832 18006 hybrid_clock.cc:648] HybridClock initialized: now 1790209059486813 us; error 35 us; skew 500 ppm
Sep 24 00:17:39 dist-test-slave-cm7b krb5kdc[16566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209059, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:39.489246 18006 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:17:39.489652 18006 webserver.cc:460] Webserver started at http://127.1.44.131:36405/ using document root <none> and password file <none>
I20260924 00:17:39.489880 18006 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:39.489924 18006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:39.491284 18006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:39.491930 18021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.492125 18006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:17:39.492208 18006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "8818e2274d3e4d0c9d826cd56b9569eb"
format_stamp: "Formatted at 2026-09-24 00:16:58 on dist-test-slave-cm7b"
server_key: "8604479ac7c4c8f6d509e572fca80bf4"
server_key_iv: "eb85c32afc9dccff3ee7954808352d46"
server_key_version: "encryptionkey@0"
I20260924 00:17:39.492534 18006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:39.503039 18006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:39.504747 18006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:39.504894 18006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:39.505092 18006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:39.505522 18028 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260924 00:17:39.506806 18028 ts_tablet_manager.cc:1852] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260924 00:17:39.507066 18028 ts_tablet_manager.cc:1905] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:39.509184 18028 ts_tablet_manager.cc:1918] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:39.509255 18028 log.cc:1196] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/c8ed489a049c4af595f424faff78d655
I20260924 00:17:39.509450 18028 ts_tablet_manager.cc:1939] T c8ed489a049c4af595f424faff78d655 P 8818e2274d3e4d0c9d826cd56b9569eb: Deleting consensus metadata
I20260924 00:17:39.509624 18006 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260924 00:17:39.509667 18006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260924 00:17:39.509690 18006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:39.509702 18006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:39.516004 18006 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:41667
I20260924 00:17:39.516055 18135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:41667 every 8 connection(s)
I20260924 00:17:39.516331 18006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:17:39.517305 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.516941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:36161)
0924 00:17:39.517111 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.517179 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
W20260924 00:17:39.517575 18136 heartbeater.cc:644] Failed to heartbeat to 127.1.44.190:35641 (0 consecutive failures): Network error: Failed to ping master at 127.1.44.190:35641: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
I20260924 00:17:39.517858 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.517773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:37805)
0924 00:17:39.517816 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.517841 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 00:17:39.518118 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:39.518047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:59497)
0924 00:17:39.518082 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:39.518105 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 00:17:39.518231 18136 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 00:17:39.522007 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18006
I20260924 00:17:40.338116 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.337847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:39145)
0924 00:17:40.338007 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:40.338049 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260924 00:17:40.397262 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.397047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:55451)
0924 00:17:40.397182 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:40.397215 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":56,"threads_started":1}
I20260924 00:17:40.518704 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.518541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:47095)
0924 00:17:40.518642 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:40.518673 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
I20260924 00:17:40.931740 18143 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:40.931854 18143 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:40.932349 18143 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
Sep 24 00:17:40 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209059, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.131@KRBTEST.COM
Sep 24 00:17:40 dist-test-slave-cm7b krb5kdc[16566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209059, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.130@KRBTEST.COM
I20260924 00:17:40.947389 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.932586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:50599 (local address 127.1.44.130:43629)
0924 00:17:40.932744 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:40.932759 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:40.932785 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:17:40.933268 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:40.933270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:40.933314 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:40.933467 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:40.934242 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.935631 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.937035 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.937639 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.937753 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:40.944023 (+ 6270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:40.944027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:40.944027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:40.944036 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:40.945828 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:40.946228 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:40.946231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:40.946231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:40.946340 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:40.946574 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:40.946574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:40.946575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:40.946810 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:40.946892 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:40.947140 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 00:17:40.947282 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260924 00:17:40.947392 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.932635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:41997 (local address 127.1.44.131:41667)
0924 00:17:40.932758 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:40.932769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:40.932817 (+ 48us) server_negotiation.cc:407] Connection header received
0924 00:17:40.933313 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:40.933315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:40.933353 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:40.933504 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:40.934568 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.935841 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.937035 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.937644 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.937753 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:40.943604 (+ 5851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:40.943609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:40.943609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:40.943622 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:40.945584 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:40.946062 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:40.946065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:40.946066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:40.946179 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:40.946437 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:40.946438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:40.946438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:40.946736 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:40.946796 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:40.947122 (+ 326us) server_negotiation.cc:299] Negotiation successful
0924 00:17:40.947277 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260924 00:17:40.947392 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.932503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:41667 (local address 127.1.44.129:41997)
0924 00:17:40.932763 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:40.932772 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:40.932819 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:40.933507 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:40.933510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:40.933910 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:40.934280 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:40.934282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.935931 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.935932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:40.936938 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:40.936940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.937746 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.937747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:40.937910 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:40.941552 (+ 3642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:40.941563 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:40.943489 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:40.945672 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:40.945679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:40.945681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:40.945976 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:40.946250 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:40.946251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:40.946251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:40.946370 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:40.946808 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:40.946815 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:40.946954 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 00:17:40.947062 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 00:17:40.947275 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":4,"thread_start_us":57,"threads_started":1}
I20260924 00:17:40.947491 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:40.932478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:43629 (local address 127.1.44.129:50599)
0924 00:17:40.932610 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:40.932621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:40.932680 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:40.933506 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:40.933509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:40.933908 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:40.934119 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:40.934124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.935739 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.935740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:40.936938 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:40.936940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:40.937745 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:40.937746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:40.937910 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:40.941552 (+ 3642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:40.941563 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:40.943853 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:40.945916 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:40.945917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:40.945917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:40.946133 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:40.946432 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:40.946432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:40.946433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:40.946509 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:40.946893 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:40.946893 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:40.946953 (+ 60us) client_negotiation.cc:769] Sending connection context
0924 00:17:40.947069 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 00:17:40.947352 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":155,"thread_start_us":63,"threads_started":1}
W20260924 00:17:40.948552 17757 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
W20260924 00:17:40.948657 17755 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
I20260924 00:17:40.948719 17755 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51e1904fd0d44c9bf5b5e5093f942bc; no voters: 0a49182fc25e49e9b606b8931f3b3871, 8818e2274d3e4d0c9d826cd56b9569eb
I20260924 00:17:40.948839 18143 raft_consensus.cc:2726] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:17:41.338829 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:41.338606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:53585)
0924 00:17:41.338749 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:41.338787 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260924 00:17:41.397869 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:41.397667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:35671)
0924 00:17:41.397785 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:41.397828 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260924 00:17:41.519438 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:41.519182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:45143)
0924 00:17:41.519370 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:41.519405 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":129,"threads_started":1}
I20260924 00:17:42.339506 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:42.339303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:56267)
0924 00:17:42.339420 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:42.339454 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260924 00:17:42.398420 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:42.398241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:52927)
0924 00:17:42.398351 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:42.398386 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260924 00:17:42.520071 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:42.519875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:35747)
0924 00:17:42.520000 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:42.520036 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260924 00:17:42.549469 18153 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:42.549556 18153 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:42.549726 18153 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
W20260924 00:17:42.550100 17755 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
W20260924 00:17:42.550185 17757 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
I20260924 00:17:42.550248 17757 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51e1904fd0d44c9bf5b5e5093f942bc; no voters: 0a49182fc25e49e9b606b8931f3b3871, 8818e2274d3e4d0c9d826cd56b9569eb
I20260924 00:17:42.550338 18153 raft_consensus.cc:2726] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:17:43.340205 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:43.340004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:57183)
0924 00:17:43.340129 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:43.340159 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260924 00:17:43.399097 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:43.398861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:46013)
0924 00:17:43.399011 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:43.399048 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260924 00:17:43.520766 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:43.520560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:34305)
0924 00:17:43.520691 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:43.520726 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260924 00:17:44.340972 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:44.340745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:48929)
0924 00:17:44.340886 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:44.340924 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260924 00:17:44.399730 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:44.399523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:48753)
0924 00:17:44.399658 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:44.399691 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260924 00:17:44.521377 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:44.521183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:36455)
0924 00:17:44.521307 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:44.521341 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":55,"threads_started":1}
I20260924 00:17:45.341651 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:45.341441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:43831)
0924 00:17:45.341574 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:45.341608 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260924 00:17:45.400342 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:45.400173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:55779)
0924 00:17:45.400278 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:45.400309 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260924 00:17:45.522024 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:45.521815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:60987)
0924 00:17:45.521958 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:45.521990 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260924 00:17:45.788707 18163 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:45.788815 18163 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:45.789036 18163 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
W20260924 00:17:45.789392 17755 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
W20260924 00:17:45.789464 17757 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
I20260924 00:17:45.789549 17757 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51e1904fd0d44c9bf5b5e5093f942bc; no voters: 0a49182fc25e49e9b606b8931f3b3871, 8818e2274d3e4d0c9d826cd56b9569eb
I20260924 00:17:45.789618 18163 raft_consensus.cc:2726] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:17:46.342383 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:46.342145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:34799)
0924 00:17:46.342300 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:46.342337 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
I20260924 00:17:46.400920 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:46.400709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:41559)
0924 00:17:46.400847 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:46.400882 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":54,"threads_started":1}
I20260924 00:17:46.522765 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:46.522507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:57705)
0924 00:17:46.522677 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:46.522722 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260924 00:17:47.343137 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:47.342907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:51643)
0924 00:17:47.343039 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:47.343082 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260924 00:17:47.401618 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:47.401398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:52147)
0924 00:17:47.401533 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:47.401575 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260924 00:17:47.523521 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:47.523312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:41301)
0924 00:17:47.523438 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:47.523474 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260924 00:17:48.343853 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:48.343641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:58437)
0924 00:17:48.343766 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:48.343805 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260924 00:17:48.402334 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:48.402099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:52533)
0924 00:17:48.402249 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:48.402289 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260924 00:17:48.490712 18172 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:48.490797 18172 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:48.490952 18172 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
W20260924 00:17:48.491354 17757 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
W20260924 00:17:48.491441 17755 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
I20260924 00:17:48.491477 17755 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51e1904fd0d44c9bf5b5e5093f942bc; no voters: 0a49182fc25e49e9b606b8931f3b3871, 8818e2274d3e4d0c9d826cd56b9569eb
I20260924 00:17:48.491562 18172 raft_consensus.cc:2726] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:17:48.524262 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:48.524020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:36985)
0924 00:17:48.524184 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:48.524221 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260924 00:17:49.344676 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:49.344392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:44929)
0924 00:17:49.344548 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:49.344589 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260924 00:17:49.403020 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:49.402819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:44999)
0924 00:17:49.402938 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:49.402977 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260924 00:17:49.524917 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:49.524724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:33923)
0924 00:17:49.524845 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:49.524876 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260924 00:17:50.345403 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:50.345207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:60287)
0924 00:17:50.345325 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:50.345356 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260924 00:17:50.403807 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:50.403600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:44427)
0924 00:17:50.403725 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:50.403761 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260924 00:17:50.525537 18179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:50.525332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:33177)
0924 00:17:50.525452 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:50.525486 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260924 00:17:51.346122 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:51.345895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:58701)
0924 00:17:51.346034 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:51.346076 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260924 00:17:51.404472 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:51.404274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:36525)
0924 00:17:51.404391 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:51.404429 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260924 00:17:51.526202 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:51.525986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:39383)
0924 00:17:51.526119 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:51.526160 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260924 00:17:52.346925 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:52.346675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:46305)
0924 00:17:52.346828 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:52.346878 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260924 00:17:52.405154 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:52.404951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:37155)
0924 00:17:52.405065 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:52.405100 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260924 00:17:52.526969 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:52.526714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:34369)
0924 00:17:52.526883 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:52.526925 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260924 00:17:53.347656 18186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:53.347439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:55149)
0924 00:17:53.347568 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:53.347608 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
I20260924 00:17:53.405861 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:53.405641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:45525)
0924 00:17:53.405779 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:53.405816 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260924 00:17:53.527669 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:53.527439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:40041)
0924 00:17:53.527584 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:53.527625 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260924 00:17:54.093338 18189 raft_consensus.cc:482] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:54.093438 18189 raft_consensus.cc:504] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a49182fc25e49e9b606b8931f3b3871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 43629 } } peers { permanent_uuid: "8818e2274d3e4d0c9d826cd56b9569eb" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 41667 } } peers { permanent_uuid: "d51e1904fd0d44c9bf5b5e5093f942bc" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 40999 } }
I20260924 00:17:54.093621 18189 leader_election.cc:288] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629), 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667)
W20260924 00:17:54.093989 17755 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0a49182fc25e49e9b606b8931f3b3871 (127.1.44.130:43629): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
W20260924 00:17:54.094074 17757 leader_election.cc:341] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8818e2274d3e4d0c9d826cd56b9569eb (127.1.44.131:41667): Not found: Tablet not found: c8ed489a049c4af595f424faff78d655
I20260924 00:17:54.094175 17757 leader_election.cc:302] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51e1904fd0d44c9bf5b5e5093f942bc; no voters: 0a49182fc25e49e9b606b8931f3b3871, 8818e2274d3e4d0c9d826cd56b9569eb
I20260924 00:17:54.094264 18189 raft_consensus.cc:2726] T c8ed489a049c4af595f424faff78d655 P d51e1904fd0d44c9bf5b5e5093f942bc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 00:17:54.348451 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:54.348197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.129:48683)
0924 00:17:54.348355 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:54.348389 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260924 00:17:54.406821 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:54.406568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.130:50011)
0924 00:17:54.406729 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:54.406772 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260924 00:17:54.527604 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 17705
I20260924 00:17:54.529291 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:54.529039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:35641 (local address 127.1.44.131:58039)
0924 00:17:54.529176 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:54.529218 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.190:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260924 00:17:54.532562 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 17874
I20260924 00:17:54.536808 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18006
2026-09-24T00:17:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58195 ms)
[----------] 30 tests from SecurityITest (228267 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:17:54 dist-test-slave-cm7b krb5kdc[18195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:17:54 dist-test-slave-cm7b krb5kdc[18195](info): set up 2 sockets
Sep 24 00:17:54 dist-test-slave-cm7b krb5kdc[18195](info): commencing operation
krb5kdc: starting...
W20260924 00:17:55.574957 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:17:55 dist-test-slave-cm7b krb5kdc[18195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:17:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:17:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:17:55.705999 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36493
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:36493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:55.786813 18208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:55.787019 18208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:55.787042 18208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:55.788168 18208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:17:55.788208 18208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:55.788223 18208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:55.788236 18208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:17:55.788249 18208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:17:55.789626 18208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:36493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:55.789815 18208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:55.789980 18208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:55.795159 18214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:55.795166 18213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:55.795243 18208 server_base.cc:1034] running on GCE node
W20260924 00:17:55.795266 18216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:55.795553 18208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:55.795799 18208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:55.796939 18208 hybrid_clock.cc:648] HybridClock initialized: now 1790209075796920 us; error 40 us; skew 500 ppm
Sep 24 00:17:55 dist-test-slave-cm7b krb5kdc[18195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:55.799418 18208 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:17:55.799865 18208 webserver.cc:460] Webserver started at http://127.1.44.190:34061/ using document root <none> and password file <none>
I20260924 00:17:55.800045 18208 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:55.800084 18208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:55.800169 18208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:55.801252 18208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "93e313cd3b3740379aaa631e91157198"
format_stamp: "Formatted at 2026-09-24 00:17:55 on dist-test-slave-cm7b"
server_key: "56149b8f717c6cc146ae3c812762d310"
server_key_iv: "847036f1a3edb0efba14d4d327e6ffde"
server_key_version: "encryptionkey@0"
I20260924 00:17:55.801551 18208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "93e313cd3b3740379aaa631e91157198"
format_stamp: "Formatted at 2026-09-24 00:17:55 on dist-test-slave-cm7b"
server_key: "56149b8f717c6cc146ae3c812762d310"
server_key_iv: "847036f1a3edb0efba14d4d327e6ffde"
server_key_version: "encryptionkey@0"
I20260924 00:17:55.802791 18208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:17:55.803493 18223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:55.803658 18208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:17:55.803769 18208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "93e313cd3b3740379aaa631e91157198"
format_stamp: "Formatted at 2026-09-24 00:17:55 on dist-test-slave-cm7b"
server_key: "56149b8f717c6cc146ae3c812762d310"
server_key_iv: "847036f1a3edb0efba14d4d327e6ffde"
server_key_version: "encryptionkey@0"
I20260924 00:17:55.803844 18208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:55.814685 18208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:55.816699 18208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:55.816866 18208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:55.820901 18208 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:36493
I20260924 00:17:55.820930 18275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:36493 every 8 connection(s)
I20260924 00:17:55.821285 18208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:17:55.822113 18276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:55.824402 18276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198: Bootstrap starting.
I20260924 00:17:55.824920 18276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:55.825145 18276 log.cc:824] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:55.825796 18276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198: No bootstrap required, opened a new log
I20260924 00:17:55.827371 18276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e313cd3b3740379aaa631e91157198" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36493 } }
I20260924 00:17:55.827505 18276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:55.827527 18276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e313cd3b3740379aaa631e91157198, State: Initialized, Role: FOLLOWER
I20260924 00:17:55.827661 18276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e313cd3b3740379aaa631e91157198" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36493 } }
I20260924 00:17:55.827739 18276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:17:55.827762 18276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:17:55.827791 18276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:55.828253 18276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e313cd3b3740379aaa631e91157198" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36493 } }
I20260924 00:17:55.828332 18276 leader_election.cc:302] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93e313cd3b3740379aaa631e91157198; no voters:
I20260924 00:17:55.828444 18276 leader_election.cc:288] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:17:55.828526 18279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:55.828722 18279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [term 1 LEADER]: Becoming Leader. State: Replica: 93e313cd3b3740379aaa631e91157198, State: Running, Role: LEADER
I20260924 00:17:55.828747 18276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:17:55.828857 18279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e313cd3b3740379aaa631e91157198" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36493 } }
I20260924 00:17:55.829316 18281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93e313cd3b3740379aaa631e91157198. Latest consensus state: current_term: 1 leader_uuid: "93e313cd3b3740379aaa631e91157198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e313cd3b3740379aaa631e91157198" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36493 } } }
I20260924 00:17:55.829440 18280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93e313cd3b3740379aaa631e91157198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e313cd3b3740379aaa631e91157198" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36493 } } }
I20260924 00:17:55.829476 18281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:55.829628 18280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:55.830129 18288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:17:55.830719 18288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:17:55.830829 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18208
I20260924 00:17:55.830961 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:17:55.831158 1202 external_mini_cluster.cc:1428] Setting key 7c3eb1a55b5646eb6c8416ab0d48f93a
I20260924 00:17:55.832353 18288 catalog_manager.cc:1347] Generated new cluster ID: 78dd330d4502487f8a97d0fafa0f73ba
I20260924 00:17:55.832403 18288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:17:55 dist-test-slave-cm7b krb5kdc[18195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:55.843649 18288 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:17:55.843909 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:55.832023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.1.44.190:36493)
0924 00:17:55.832211 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:55.832222 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:55.832257 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:17:55.832383 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:55.832390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:55.832453 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:55.832608 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:55.833325 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:55.834924 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:55.836005 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:55.836735 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:55.836859 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:55.839859 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:55.839864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:55.839864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:55.839877 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:55.841877 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:55.842315 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:55.842318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:55.842318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:55.842434 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:55.842672 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:55.842672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:55.842673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:55.842971 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:55.843144 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:55.843486 (+ 342us) server_negotiation.cc:299] Negotiation successful
0924 00:17:55.843635 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260924 00:17:55.844609 18288 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:17:55.853219 18288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198: Generated new TSK 0
I20260924 00:17:55.853433 18288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:17:55.882207 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36493
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:55.962878 18302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:55.963083 18302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:55.963107 18302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:55.964299 18302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:55.964344 18302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:55.964370 18302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:17:55.965809 18302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:55.966054 18302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:55.966305 18302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:55.971298 18310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:55.971299 18307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:55.971510 18302 server_base.cc:1034] running on GCE node
W20260924 00:17:55.971379 18308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:55.971778 18302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:55.972028 18302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:55.973194 18302 hybrid_clock.cc:648] HybridClock initialized: now 1790209075973189 us; error 35 us; skew 500 ppm
Sep 24 00:17:55 dist-test-slave-cm7b krb5kdc[18195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:55.975669 18302 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:17:55.976094 18302 webserver.cc:460] Webserver started at http://127.1.44.129:38071/ using document root <none> and password file <none>
I20260924 00:17:55.976317 18302 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:55.976387 18302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:55.976550 18302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:55.977577 18302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "783072ce15f24f9d950d6fe6755d5150"
format_stamp: "Formatted at 2026-09-24 00:17:55 on dist-test-slave-cm7b"
server_key: "c22a16e6a441fa780ac253d02be3de69"
server_key_iv: "f347c6e6a4ed5298b2ce63f04b299124"
server_key_version: "encryptionkey@0"
I20260924 00:17:55.977941 18302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "783072ce15f24f9d950d6fe6755d5150"
format_stamp: "Formatted at 2026-09-24 00:17:55 on dist-test-slave-cm7b"
server_key: "c22a16e6a441fa780ac253d02be3de69"
server_key_iv: "f347c6e6a4ed5298b2ce63f04b299124"
server_key_version: "encryptionkey@0"
I20260924 00:17:55.979328 18302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:17:55.980077 18317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:55.980285 18302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:55.980392 18302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "783072ce15f24f9d950d6fe6755d5150"
format_stamp: "Formatted at 2026-09-24 00:17:55 on dist-test-slave-cm7b"
server_key: "c22a16e6a441fa780ac253d02be3de69"
server_key_iv: "f347c6e6a4ed5298b2ce63f04b299124"
server_key_version: "encryptionkey@0"
I20260924 00:17:55.980480 18302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.006841 18302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.009169 18302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.009348 18302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.009617 18302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:56.009922 18302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:56.009975 18302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.010016 18302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:56.010046 18302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.015763 18302 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:36525
I20260924 00:17:56.015803 18430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:36525 every 8 connection(s)
I20260924 00:17:56.016114 18302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:17:56.017133 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18302
I20260924 00:17:56.017254 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:17:56.017438 1202 external_mini_cluster.cc:1428] Setting key e8003ccc8e6bd05220e879fa01c9f443
Sep 24 00:17:56 dist-test-slave-cm7b krb5kdc[18195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209075, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:17:56.025594 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.016751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:48923 (local address 127.1.44.190:36493)
0924 00:17:56.016794 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.016796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.017273 (+ 477us) server_negotiation.cc:407] Connection header received
0924 00:17:56.017593 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.017595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.017611 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.017664 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.018722 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.019326 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.020312 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.020533 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.020611 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.022738 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.022740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.022741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:56.022748 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.024198 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.024623 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.024624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.024625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.024684 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.024939 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.024940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.024940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.025054 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.025132 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.025403 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.025502 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:17:56.025897 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.016748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:36493 (local address 127.1.44.129:48923)
0924 00:17:56.016895 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.016930 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.017476 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.017799 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.017802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.018429 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:56.018627 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.018631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.019418 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.019418 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.020217 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.020219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.020629 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.020630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.020753 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.021284 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:56.021291 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:56.022644 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:56.024296 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:56.024303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:56.024305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:56.024527 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:56.024753 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:56.024754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:56.024754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:56.024862 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:56.025132 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:56.025140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:56.025275 (+ 135us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.025382 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.025536 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260924 00:17:56.026476 18431 heartbeater.cc:347] Connected to a master server at 127.1.44.190:36493
I20260924 00:17:56.026619 18431 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:56.026805 18431 heartbeater.cc:511] Master 127.1.44.190:36493 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
I20260924 00:17:56.027381 18240 ts_manager.cc:194] Registered new tserver with Master: 783072ce15f24f9d950d6fe6755d5150 (127.1.44.129:36525)
I20260924 00:17:56.028293 18240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:48923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:17:56.042443 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36493
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:56.121289 18436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:56.121490 18436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:56.121513 18436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:56.122579 18436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:56.122615 18436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:56.122641 18436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:17:56.124006 18436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:56.124194 18436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:56.124367 18436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:56.129331 18444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.129256 18442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.129284 18441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.129518 18436 server_base.cc:1034] running on GCE node
I20260924 00:17:56.129855 18436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:56.130154 18436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:56.131309 18436 hybrid_clock.cc:648] HybridClock initialized: now 1790209076131286 us; error 27 us; skew 500 ppm
Sep 24 00:17:56 dist-test-slave-cm7b krb5kdc[18195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:56.133698 18436 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:17:56.134087 18436 webserver.cc:460] Webserver started at http://127.1.44.130:45881/ using document root <none> and password file <none>
I20260924 00:17:56.134282 18436 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:56.134323 18436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:56.134411 18436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:56.135492 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "ea2c6de6a1e742e28c2cdfc733c7444e"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "5c28aa2ba14abfa4f4b133a221c7d365"
server_key_iv: "bc7419ccc156548d852ba4e33272a72d"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.135797 18436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "ea2c6de6a1e742e28c2cdfc733c7444e"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "5c28aa2ba14abfa4f4b133a221c7d365"
server_key_iv: "bc7419ccc156548d852ba4e33272a72d"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.137092 18436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:17:56.137826 18451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.138017 18436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:56.138116 18436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "ea2c6de6a1e742e28c2cdfc733c7444e"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "5c28aa2ba14abfa4f4b133a221c7d365"
server_key_iv: "bc7419ccc156548d852ba4e33272a72d"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.138200 18436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.157423 18436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.159477 18436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.159653 18436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.159891 18436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:56.160205 18436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:56.160262 18436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.160298 18436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:56.160326 18436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.166182 18436 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:39015
I20260924 00:17:56.166230 18564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:39015 every 8 connection(s)
I20260924 00:17:56.166548 18436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:17:56.167094 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18436
I20260924 00:17:56.167237 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:17:56.167480 1202 external_mini_cluster.cc:1428] Setting key 760280018b60958ede9b19880bedf94f
Sep 24 00:17:56 dist-test-slave-cm7b krb5kdc[18195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
I20260924 00:17:56.177229 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.167235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:58851 (local address 127.1.44.190:36493)
0924 00:17:56.167282 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.167284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.167594 (+ 310us) server_negotiation.cc:407] Connection header received
0924 00:17:56.168212 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.168213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.168226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.168271 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.169611 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.170118 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.171702 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.171901 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.172011 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.174166 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.174167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.174168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:56.174176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.175737 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.176224 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.176226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.176226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.176285 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.176534 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.176534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.176535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.176651 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.176774 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.177045 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.177133 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:17:56.177423 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.167249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:36493 (local address 127.1.44.130:58851)
0924 00:17:56.167428 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.167480 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.168134 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.168353 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.168357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.169178 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:56.169462 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.169468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.170213 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.170214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.171592 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.171595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.172016 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.172018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.172158 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.172700 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:56.172708 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:56.174066 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:56.175841 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:56.175849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:56.175851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:56.176138 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:56.176352 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:56.176353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:56.176353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:56.176464 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:56.176766 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:56.176772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:56.176914 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.177026 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.177183 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260924 00:17:56.177901 18565 heartbeater.cc:347] Connected to a master server at 127.1.44.190:36493
I20260924 00:17:56.178010 18565 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:56.178182 18565 heartbeater.cc:511] Master 127.1.44.190:36493 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
I20260924 00:17:56.178588 18240 ts_manager.cc:194] Registered new tserver with Master: ea2c6de6a1e742e28c2cdfc733c7444e (127.1.44.130:39015)
I20260924 00:17:56.178995 18240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:58851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:17:56.194197 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36493
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:56.274942 18570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:56.275144 18570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:56.275167 18570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:56.276338 18570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:56.276381 18570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:56.276407 18570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:17:56.277797 18570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:56.277981 18570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:56.278148 18570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:56.283116 18575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.283094 18576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.283106 18578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.283459 18570 server_base.cc:1034] running on GCE node
I20260924 00:17:56.283656 18570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:56.283921 18570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:56.285074 18570 hybrid_clock.cc:648] HybridClock initialized: now 1790209076285070 us; error 35 us; skew 500 ppm
Sep 24 00:17:56 dist-test-slave-cm7b krb5kdc[18195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:56.287586 18570 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:17:56.288040 18570 webserver.cc:460] Webserver started at http://127.1.44.131:39563/ using document root <none> and password file <none>
I20260924 00:17:56.288275 18570 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:56.288342 18570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:56.288492 18570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:56.289498 18570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "7cbf83d073d94bc796abee2f50c4236a"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "78120fdce4af65126d0d027a99950b0c"
server_key_iv: "fd021c85a10a8f8215a794c4c8c93c3d"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.289837 18570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "7cbf83d073d94bc796abee2f50c4236a"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "78120fdce4af65126d0d027a99950b0c"
server_key_iv: "fd021c85a10a8f8215a794c4c8c93c3d"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.291093 18570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:17:56.291893 18585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.292081 18570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:56.292184 18570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "7cbf83d073d94bc796abee2f50c4236a"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "78120fdce4af65126d0d027a99950b0c"
server_key_iv: "fd021c85a10a8f8215a794c4c8c93c3d"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.292263 18570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.321672 18570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.324177 18570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.324342 18570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.324570 18570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:56.324896 18570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:56.324952 18570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.324993 18570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:56.325021 18570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.331067 18570 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:40219
I20260924 00:17:56.331161 18698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:40219 every 8 connection(s)
I20260924 00:17:56.331472 18570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
Sep 24 00:17:56 dist-test-slave-cm7b krb5kdc[18195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209076, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:56.339318 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18570
I20260924 00:17:56.339458 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:17:56.339691 1202 external_mini_cluster.cc:1428] Setting key 523825f6ce854f3847272850b3bf2126
I20260924 00:17:56.340474 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.332212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:57559 (local address 127.1.44.190:36493)
0924 00:17:56.332280 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.332282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.332469 (+ 187us) server_negotiation.cc:407] Connection header received
0924 00:17:56.332936 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.332937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.332946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.333006 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.333901 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.334386 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.335452 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.335637 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.335693 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.337731 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.337733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.337733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:56.337739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.338899 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.339411 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.339412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.339412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.339474 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.339786 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.339787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.339787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.339935 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.340005 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.340334 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.340411 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 00:17:56.340718 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.332198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:36493 (local address 127.1.44.131:57559)
0924 00:17:56.332342 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.332383 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.332857 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.333075 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.333078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.333635 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:56.333828 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.333833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.334469 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.334470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.335376 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.335379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.335711 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.335712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.335816 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.336336 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:56.336344 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:56.337645 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:56.338969 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:56.338975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:56.338976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:56.339320 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:56.339560 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:56.339562 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:56.339562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:56.339713 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:56.340005 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:56.340011 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:56.340171 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.340304 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.340471 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260924 00:17:56.341156 18699 heartbeater.cc:347] Connected to a master server at 127.1.44.190:36493
I20260924 00:17:56.341269 18699 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:56.341441 18699 heartbeater.cc:511] Master 127.1.44.190:36493 requested a full tablet report, sending...
I20260924 00:17:56.341784 18240 ts_manager.cc:194] Registered new tserver with Master: 7cbf83d073d94bc796abee2f50c4236a (127.1.44.131:40219)
I20260924 00:17:56.342182 18240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:57559
I20260924 00:17:56.352087 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:17:56.359623 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.353490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47434 (local address 127.1.44.190:36493)
0924 00:17:56.353567 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.353569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.353675 (+ 106us) server_negotiation.cc:407] Connection header received
0924 00:17:56.353766 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.353767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.353777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.353835 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.354388 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.354889 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.355845 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.356055 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.356143 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.357290 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.357292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.357292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:56.357300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.358429 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.358760 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.358761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.358762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.358818 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:56.359020 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:56.359021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:56.359021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:56.359175 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.359338 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.359440 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.359519 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 00:17:56.361058 18240 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:17:56.362547 18240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:17:56.368650 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.364766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:40219 (local address 127.0.0.1:33038)
0924 00:17:56.365222 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.365232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.365295 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.365581 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.365585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.365588 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:56.365667 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.365669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.366551 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.366552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.367498 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.367500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.368345 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.368346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.368427 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.368438 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.368506 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.368555 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":9,"thread_start_us":63,"threads_started":1}
I20260924 00:17:56.368678 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.364867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.1.44.131:40219)
0924 00:17:56.365137 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.365143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.365287 (+ 144us) server_negotiation.cc:407] Connection header received
0924 00:17:56.365367 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.365370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.365389 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.365468 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:56.365768 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.366451 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.367594 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.368187 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.368226 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.368266 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.368549 (+ 283us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.368593 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":94,"threads_started":1}
I20260924 00:17:56.369328 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.364766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39015 (local address 127.0.0.1:37164)
0924 00:17:56.365111 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.365153 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.365233 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.365585 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.365586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.365589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:56.365664 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.365666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.367009 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.367010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.368234 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.368237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.369073 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.369073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.369141 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.369145 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.369222 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.369261 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":33,"thread_start_us":79,"threads_started":1}
I20260924 00:17:56.369352 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.364970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.1.44.130:39015)
0924 00:17:56.365154 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.365160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.365219 (+ 59us) server_negotiation.cc:407] Connection header received
0924 00:17:56.365396 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.365399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.365415 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.365470 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:56.365753 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.366880 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.368322 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.368977 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.369060 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.369113 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.369238 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.369278 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260924 00:17:56.369586 18633 tablet_service.cc:1467] Processing CreateTablet for tablet 09f0ed26738449e4bf903fb7f404f3dd (DEFAULT_TABLE table=test-table [id=766281b3624c46d98cfa6ddc269835c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:56.369730 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.364975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36525 (local address 127.0.0.1:55910)
0924 00:17:56.365391 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.365399 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.365451 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.365698 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.365699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.365701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:56.365752 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.365753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.367027 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.367028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.368234 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.368237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.369500 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.369501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.369576 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.369581 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.369635 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.369672 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":48,"thread_start_us":69,"threads_started":1}
I20260924 00:17:56.369973 18633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f0ed26738449e4bf903fb7f404f3dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:56.369980 18499 tablet_service.cc:1467] Processing CreateTablet for tablet 09f0ed26738449e4bf903fb7f404f3dd (DEFAULT_TABLE table=test-table [id=766281b3624c46d98cfa6ddc269835c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:56.370092 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.365105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55910 (local address 127.1.44.129:36525)
0924 00:17:56.365302 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.365306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.365490 (+ 184us) server_negotiation.cc:407] Connection header received
0924 00:17:56.365536 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.365538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.365554 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.365624 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:56.365845 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.366915 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.368336 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.369221 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.369288 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.369340 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.369925 (+ 585us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.369968 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260924 00:17:56.370296 18499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f0ed26738449e4bf903fb7f404f3dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:56.370712 18365 tablet_service.cc:1467] Processing CreateTablet for tablet 09f0ed26738449e4bf903fb7f404f3dd (DEFAULT_TABLE table=test-table [id=766281b3624c46d98cfa6ddc269835c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:56.370998 18365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09f0ed26738449e4bf903fb7f404f3dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:56.373129 18716 tablet_bootstrap.cc:492] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Bootstrap starting.
I20260924 00:17:56.373558 18717 tablet_bootstrap.cc:492] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Bootstrap starting.
I20260924 00:17:56.373844 18716 tablet_bootstrap.cc:654] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:56.374184 18718 tablet_bootstrap.cc:492] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150: Bootstrap starting.
I20260924 00:17:56.374325 18717 tablet_bootstrap.cc:654] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:56.374485 18716 log.cc:824] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:56.374872 18718 tablet_bootstrap.cc:654] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:56.374915 18717 log.cc:824] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:56.375377 18716 tablet_bootstrap.cc:492] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: No bootstrap required, opened a new log
I20260924 00:17:56.375465 18716 ts_tablet_manager.cc:1397] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:56.375479 18718 log.cc:824] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:56.375715 18717 tablet_bootstrap.cc:492] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: No bootstrap required, opened a new log
I20260924 00:17:56.375814 18717 ts_tablet_manager.cc:1397] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:56.376418 18718 tablet_bootstrap.cc:492] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150: No bootstrap required, opened a new log
I20260924 00:17:56.376488 18718 ts_tablet_manager.cc:1397] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:56.377209 18716 raft_consensus.cc:348] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.377382 18716 raft_consensus.cc:374] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:56.377427 18716 raft_consensus.cc:729] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cbf83d073d94bc796abee2f50c4236a, State: Initialized, Role: FOLLOWER
I20260924 00:17:56.377556 18716 consensus_queue.cc:260] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.377831 18716 ts_tablet_manager.cc:1428] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:17:56.377974 18699 heartbeater.cc:503] Master 127.1.44.190:36493 was elected leader, sending a full tablet report...
I20260924 00:17:56.377875 18717 raft_consensus.cc:348] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.378055 18717 raft_consensus.cc:374] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:56.378088 18717 raft_consensus.cc:729] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2c6de6a1e742e28c2cdfc733c7444e, State: Initialized, Role: FOLLOWER
I20260924 00:17:56.378219 18717 consensus_queue.cc:260] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.378491 18717 ts_tablet_manager.cc:1428] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:17:56.378512 18565 heartbeater.cc:503] Master 127.1.44.190:36493 was elected leader, sending a full tablet report...
I20260924 00:17:56.378594 18718 raft_consensus.cc:348] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.378793 18718 raft_consensus.cc:374] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:56.378832 18718 raft_consensus.cc:729] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783072ce15f24f9d950d6fe6755d5150, State: Initialized, Role: FOLLOWER
I20260924 00:17:56.378939 18718 consensus_queue.cc:260] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.379252 18718 ts_tablet_manager.cc:1428] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:17:56.379256 18431 heartbeater.cc:503] Master 127.1.44.190:36493 was elected leader, sending a full tablet report...
I20260924 00:17:56.390585 18722 raft_consensus.cc:482] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:56.390730 18722 raft_consensus.cc:504] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.391095 18722 leader_election.cc:288] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea2c6de6a1e742e28c2cdfc733c7444e (127.1.44.130:39015), 783072ce15f24f9d950d6fe6755d5150 (127.1.44.129:36525)
I20260924 00:17:56.394027 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.391288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36525 (local address 127.1.44.131:38045)
0924 00:17:56.391343 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.391355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.391399 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.391556 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.391557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.391559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:56.391637 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.391639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.392366 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.392367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.393193 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.393196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.393777 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.393778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.393859 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.393864 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.393919 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.393965 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1}
I20260924 00:17:56.394047 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.391328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:38045 (local address 127.1.44.129:36525)
0924 00:17:56.391376 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.391379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.391390 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:17:56.391457 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.391458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.391470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.391529 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:56.391732 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.392288 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.393282 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.393687 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.393744 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.393751 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.393935 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.393969 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:17:56.394212 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.391334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:45073 (local address 127.1.44.130:39015)
0924 00:17:56.391379 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.391381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.391633 (+ 252us) server_negotiation.cc:407] Connection header received
0924 00:17:56.391661 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.391662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.391674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.391732 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:56.391930 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.392450 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.393407 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.393837 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.393914 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.393923 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.394090 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.394134 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:17:56.394381 18385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f0ed26738449e4bf903fb7f404f3dd" candidate_uuid: "7cbf83d073d94bc796abee2f50c4236a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783072ce15f24f9d950d6fe6755d5150" is_pre_election: true
I20260924 00:17:56.394521 18385 raft_consensus.cc:2454] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cbf83d073d94bc796abee2f50c4236a in term 0.
I20260924 00:17:56.394744 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.391323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:39015 (local address 127.1.44.131:45073)
0924 00:17:56.391500 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.391526 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.391572 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.391775 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.391775 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.391777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:56.391837 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.391838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.392531 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.392532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.393308 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.393311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.393940 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.393941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.394005 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.394009 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.394066 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.394104 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
I20260924 00:17:56.394757 18589 leader_election.cc:302] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783072ce15f24f9d950d6fe6755d5150, 7cbf83d073d94bc796abee2f50c4236a; no voters:
I20260924 00:17:56.394920 18722 raft_consensus.cc:2781] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:17:56.394968 18722 raft_consensus.cc:482] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:17:56.394987 18722 raft_consensus.cc:3037] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:56.395090 18519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f0ed26738449e4bf903fb7f404f3dd" candidate_uuid: "7cbf83d073d94bc796abee2f50c4236a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" is_pre_election: true
I20260924 00:17:56.395190 18519 raft_consensus.cc:2454] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cbf83d073d94bc796abee2f50c4236a in term 0.
I20260924 00:17:56.395817 18722 raft_consensus.cc:504] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.395947 18722 leader_election.cc:288] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [CANDIDATE]: Term 1 election: Requested vote from peers ea2c6de6a1e742e28c2cdfc733c7444e (127.1.44.130:39015), 783072ce15f24f9d950d6fe6755d5150 (127.1.44.129:36525)
I20260924 00:17:56.396108 18385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f0ed26738449e4bf903fb7f404f3dd" candidate_uuid: "7cbf83d073d94bc796abee2f50c4236a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "783072ce15f24f9d950d6fe6755d5150"
I20260924 00:17:56.396130 18519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09f0ed26738449e4bf903fb7f404f3dd" candidate_uuid: "7cbf83d073d94bc796abee2f50c4236a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e"
I20260924 00:17:56.396220 18385 raft_consensus.cc:3037] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:56.396215 18519 raft_consensus.cc:3037] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:56.396911 18519 raft_consensus.cc:2454] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cbf83d073d94bc796abee2f50c4236a in term 1.
I20260924 00:17:56.396930 18385 raft_consensus.cc:2454] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cbf83d073d94bc796abee2f50c4236a in term 1.
I20260924 00:17:56.397092 18588 leader_election.cc:302] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cbf83d073d94bc796abee2f50c4236a, ea2c6de6a1e742e28c2cdfc733c7444e; no voters:
I20260924 00:17:56.397212 18722 raft_consensus.cc:2781] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:56.397352 18722 raft_consensus.cc:686] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 1 LEADER]: Becoming Leader. State: Replica: 7cbf83d073d94bc796abee2f50c4236a, State: Running, Role: LEADER
I20260924 00:17:56.397452 18722 consensus_queue.cc:237] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } }
I20260924 00:17:56.398288 18240 catalog_manager.cc:5495] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cbf83d073d94bc796abee2f50c4236a (127.1.44.131). New cstate: current_term: 1 leader_uuid: "7cbf83d073d94bc796abee2f50c4236a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cbf83d073d94bc796abee2f50c4236a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 } health_report { overall_health: UNKNOWN } } }
I20260924 00:17:56.403445 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.401251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.1.44.131:40219)
0924 00:17:56.401310 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.401311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.401398 (+ 87us) server_negotiation.cc:407] Connection header received
0924 00:17:56.401451 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.401452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.401466 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.401527 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:17:56.401605 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.402029 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.402700 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.402868 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.402916 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.403098 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:17:56.403187 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:17:56.403346 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.403367 (+ 21us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.403383 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:17:56.405031 18385 raft_consensus.cc:1261] T 09f0ed26738449e4bf903fb7f404f3dd P 783072ce15f24f9d950d6fe6755d5150 [term 1 FOLLOWER]: Refusing update from remote peer 7cbf83d073d94bc796abee2f50c4236a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:56.405063 18519 raft_consensus.cc:1261] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 1 FOLLOWER]: Refusing update from remote peer 7cbf83d073d94bc796abee2f50c4236a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:56.405355 18722 consensus_queue.cc:1035] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [LEADER]: Connected to new peer: Peer: permanent_uuid: "783072ce15f24f9d950d6fe6755d5150" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:56.405493 18726 consensus_queue.cc:1035] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2c6de6a1e742e28c2cdfc733c7444e" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 39015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:56.407756 18729 mvcc.cc:205] Tried to move back new op lower bound from 7332696376952619008 to 7332696376924876800. Current Snapshot: MvccSnapshot[applied={T|T < 7332696376952619008}]
I20260924 00:17:56.408330 18730 mvcc.cc:205] Tried to move back new op lower bound from 7332696376952619008 to 7332696376924876800. Current Snapshot: MvccSnapshot[applied={T|T < 7332696376952619008}]
I20260924 00:17:56.408349 18731 mvcc.cc:205] Tried to move back new op lower bound from 7332696376952619008 to 7332696376924876800. Current Snapshot: MvccSnapshot[applied={T|T < 7332696376952619008}]
I20260924 00:17:56.409981 18240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:56.410069 18240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:56.410822 18240 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93e313cd3b3740379aaa631e91157198: Sending DeleteTablet for 3 replicas of tablet 09f0ed26738449e4bf903fb7f404f3dd
I20260924 00:17:56.411123 18633 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f0ed26738449e4bf903fb7f404f3dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:56 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:33038
I20260924 00:17:56.411145 18499 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f0ed26738449e4bf903fb7f404f3dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:56 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:37164
I20260924 00:17:56.411393 18738 tablet_replica.cc:330] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: stopping tablet replica
I20260924 00:17:56.411408 18365 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f0ed26738449e4bf903fb7f404f3dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:56 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:55910
I20260924 00:17:56.411499 18738 raft_consensus.cc:2229] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:17:56.411586 18738 raft_consensus.cc:2258] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:56.411984 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18302
I20260924 00:17:56.413328 18738 ts_tablet_manager.cc:1905] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:56.414713 18739 tablet_replica.cc:330] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: stopping tablet replica
I20260924 00:17:56.414839 18739 raft_consensus.cc:2229] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:56.414947 18739 raft_consensus.cc:2258] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:56.415320 18738 ts_tablet_manager.cc:1918] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:56.415391 18738 log.cc:1196] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/09f0ed26738449e4bf903fb7f404f3dd
I20260924 00:17:56.415427 18739 ts_tablet_manager.cc:1905] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:56.415665 18738 ts_tablet_manager.cc:1939] T 09f0ed26738449e4bf903fb7f404f3dd P 7cbf83d073d94bc796abee2f50c4236a: Deleting consensus metadata
I20260924 00:17:56.416185 18227 catalog_manager.cc:4841] TS 7cbf83d073d94bc796abee2f50c4236a (127.1.44.131:40219): tablet 09f0ed26738449e4bf903fb7f404f3dd (table test-table [id=766281b3624c46d98cfa6ddc269835c8]) successfully deleted
I20260924 00:17:56.416838 18739 ts_tablet_manager.cc:1918] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:56.416893 18739 log.cc:1196] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/09f0ed26738449e4bf903fb7f404f3dd
I20260924 00:17:56.417141 18739 ts_tablet_manager.cc:1939] T 09f0ed26738449e4bf903fb7f404f3dd P ea2c6de6a1e742e28c2cdfc733c7444e: Deleting consensus metadata
I20260924 00:17:56.417536 18226 catalog_manager.cc:4841] TS ea2c6de6a1e742e28c2cdfc733c7444e (127.1.44.130:39015): tablet 09f0ed26738449e4bf903fb7f404f3dd (table test-table [id=766281b3624c46d98cfa6ddc269835c8]) successfully deleted
W20260924 00:17:56.422324 18227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:36525 (error 108)
I20260924 00:17:56.422508 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18436
I20260924 00:17:56.423610 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.423487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:36525 (local address 127.0.0.1:55926)
0924 00:17:56.423538 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.423574 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:36525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 00:17:56.423825 18227 catalog_manager.cc:4584] TS 783072ce15f24f9d950d6fe6755d5150 (127.1.44.129:36525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09f0ed26738449e4bf903fb7f404f3dd: Network error: Client connection negotiation failed: client connection to 127.1.44.129:36525: connect: Connection refused (error 111)
I20260924 00:17:56.427467 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18570
I20260924 00:17:56.432405 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18208
2026-09-24T00:17:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1893 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T00:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:17:56Z Disabled control of system clock
I20260924 00:17:56.459554 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:33273
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:33273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:56.538239 18745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:56.538451 18745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:56.538470 18745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:56.539554 18745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:17:56.539589 18745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:56.539602 18745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:56.539620 18745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:17:56.539633 18745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:17:56.540874 18745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:33273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:33273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:56.541056 18745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:56.541280 18745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:56.546025 18753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.546087 18751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.546222 18745 server_base.cc:1034] running on GCE node
W20260924 00:17:56.546025 18750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.546532 18745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:56.546814 18745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:56.547953 18745 hybrid_clock.cc:648] HybridClock initialized: now 1790209076547911 us; error 57 us; skew 500 ppm
I20260924 00:17:56.549105 18745 webserver.cc:460] Webserver started at http://127.1.44.190:38889/ using document root <none> and password file <none>
I20260924 00:17:56.549351 18745 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:56.549397 18745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:56.549549 18745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:56.550591 18745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "9873c8dee719415a8abfb2a427af354e"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "5193d14fa5f7dbb1cb50020c29407f7e"
server_key_iv: "5d71f61491f8dbaf7f1aed5fcd305999"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.550953 18745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "9873c8dee719415a8abfb2a427af354e"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "5193d14fa5f7dbb1cb50020c29407f7e"
server_key_iv: "5d71f61491f8dbaf7f1aed5fcd305999"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.552291 18745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:17:56.552983 18759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.553192 18745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:17:56.553290 18745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "9873c8dee719415a8abfb2a427af354e"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "5193d14fa5f7dbb1cb50020c29407f7e"
server_key_iv: "5d71f61491f8dbaf7f1aed5fcd305999"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.553360 18745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.576236 18745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.576577 18745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.576728 18745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.580909 18745 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:33273
I20260924 00:17:56.580940 18811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:33273 every 8 connection(s)
I20260924 00:17:56.581297 18745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:17:56.582278 18812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:56.583475 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18745
I20260924 00:17:56.583585 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:17:56.583777 1202 external_mini_cluster.cc:1428] Setting key 7bb9fb658fddf19be17a2826036a5554
I20260924 00:17:56.585134 18812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e: Bootstrap starting.
I20260924 00:17:56.585820 18812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:56.586139 18812 log.cc:824] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:56.586862 18812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e: No bootstrap required, opened a new log
I20260924 00:17:56.589187 18812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9873c8dee719415a8abfb2a427af354e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33273 } }
I20260924 00:17:56.589360 18812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:56.589399 18812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9873c8dee719415a8abfb2a427af354e, State: Initialized, Role: FOLLOWER
I20260924 00:17:56.589542 18812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9873c8dee719415a8abfb2a427af354e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33273 } }
I20260924 00:17:56.589627 18812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:17:56.589661 18812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:17:56.589712 18812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:56.589879 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.584323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.1.44.190:33273)
0924 00:17:56.584537 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.584547 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.584588 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:17:56.585093 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.585103 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.585185 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.585334 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.585502 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.586821 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.587665 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.588286 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.588420 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.589108 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.589112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.589112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:56.589325 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.589368 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:56.589383 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.589449 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.589557 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.589644 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260924 00:17:56.590353 18812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9873c8dee719415a8abfb2a427af354e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33273 } }
I20260924 00:17:56.590456 18812 leader_election.cc:302] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9873c8dee719415a8abfb2a427af354e; no voters:
I20260924 00:17:56.590615 18812 leader_election.cc:288] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:17:56.590663 18817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:56.590806 18817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [term 1 LEADER]: Becoming Leader. State: Replica: 9873c8dee719415a8abfb2a427af354e, State: Running, Role: LEADER
I20260924 00:17:56.590868 18812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:17:56.590894 18817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9873c8dee719415a8abfb2a427af354e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33273 } }
I20260924 00:17:56.591308 18819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9873c8dee719415a8abfb2a427af354e. Latest consensus state: current_term: 1 leader_uuid: "9873c8dee719415a8abfb2a427af354e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9873c8dee719415a8abfb2a427af354e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33273 } } }
I20260924 00:17:56.591378 18819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:56.591284 18818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9873c8dee719415a8abfb2a427af354e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9873c8dee719415a8abfb2a427af354e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 33273 } } }
I20260924 00:17:56.591518 18818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:56.591600 18825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:17:56.592054 18825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:17:56.593519 18825 catalog_manager.cc:1347] Generated new cluster ID: 6e991cdf2f3a402d81d1b3699028be2b
I20260924 00:17:56.593583 18825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:17:56.615042 18825 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:17:56.615803 18825 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:17:56.626497 18825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e: Generated new TSK 0
I20260924 00:17:56.626713 18825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:17:56.628559 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33273
--builtin_ntp_servers=127.1.44.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:56.708642 18836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:56.708835 18836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:56.708858 18836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:56.709982 18836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:56.710021 18836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:56.710045 18836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:17:56.711397 18836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:56.711637 18836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:56.711925 18836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:56.716725 18841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.716735 18842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.716739 18844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.717079 18836 server_base.cc:1034] running on GCE node
I20260924 00:17:56.717272 18836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:56.717518 18836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:56.718669 18836 hybrid_clock.cc:648] HybridClock initialized: now 1790209076718654 us; error 31 us; skew 500 ppm
I20260924 00:17:56.719879 18836 webserver.cc:460] Webserver started at http://127.1.44.129:37371/ using document root <none> and password file <none>
I20260924 00:17:56.720124 18836 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:56.720170 18836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:56.720312 18836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:56.721343 18836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "b97f138b71194191ac2cd49eb4fc840f"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "fbdda439157eb08886a7d4501f2142cc"
server_key_iv: "5a92e82e1cfde7905e3d95eaae541c2f"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.721683 18836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "b97f138b71194191ac2cd49eb4fc840f"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "fbdda439157eb08886a7d4501f2142cc"
server_key_iv: "5a92e82e1cfde7905e3d95eaae541c2f"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.722961 18836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:17:56.723726 18850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.723896 18836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.723995 18836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "b97f138b71194191ac2cd49eb4fc840f"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "fbdda439157eb08886a7d4501f2142cc"
server_key_iv: "5a92e82e1cfde7905e3d95eaae541c2f"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.724076 18836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.747041 18836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.747386 18836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.747534 18836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.747753 18836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:56.748065 18836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:56.748123 18836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.748164 18836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:56.748190 18836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.754122 18836 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:41475
I20260924 00:17:56.754161 18963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:41475 every 8 connection(s)
I20260924 00:17:56.754484 18836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:17:56.759584 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.755244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:54329 (local address 127.1.44.190:33273)
0924 00:17:56.755331 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.755334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.755612 (+ 278us) server_negotiation.cc:407] Connection header received
0924 00:17:56.756140 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.756142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.756155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.756220 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.756854 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.757412 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.758328 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.758506 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.758552 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.759359 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.759360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.759360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:56.759364 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.759376 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:56.759379 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.759424 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.759504 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.759529 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 00:17:56.759819 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.755218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33273 (local address 127.1.44.129:54329)
0924 00:17:56.755427 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.755460 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.756042 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.756272 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.756275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.756282 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:56.756780 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.756790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.757488 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.757489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.758256 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.758258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.758572 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.758572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.758679 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.759169 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:17:56.759169 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:56.759247 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:17:56.759279 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:56.759434 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:56.759438 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:56.759441 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.759492 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.759568 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
I20260924 00:17:56.760351 18964 heartbeater.cc:347] Connected to a master server at 127.1.44.190:33273
I20260924 00:17:56.760454 18964 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:56.760644 18964 heartbeater.cc:511] Master 127.1.44.190:33273 requested a full tablet report, sending...
I20260924 00:17:56.761108 18776 ts_manager.cc:194] Registered new tserver with Master: b97f138b71194191ac2cd49eb4fc840f (127.1.44.129:41475)
I20260924 00:17:56.761699 18776 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.129:54329
I20260924 00:17:56.763476 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18836
I20260924 00:17:56.763571 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:17:56.763790 1202 external_mini_cluster.cc:1428] Setting key d1f78e133f549aa2ac8dfe7a350b68e6
I20260924 00:17:56.764621 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33273
--builtin_ntp_servers=127.1.44.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:56.843550 18967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:56.843741 18967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:56.843762 18967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:56.844880 18967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:56.844919 18967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:56.844944 18967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:17:56.846235 18967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:56.846462 18967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:56.846745 18967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:56.851531 18972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.851585 18973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.851670 18967 server_base.cc:1034] running on GCE node
W20260924 00:17:56.851521 18975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.852033 18967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:56.852285 18967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:56.853456 18967 hybrid_clock.cc:648] HybridClock initialized: now 1790209076853450 us; error 37 us; skew 500 ppm
I20260924 00:17:56.854789 18967 webserver.cc:460] Webserver started at http://127.1.44.130:38761/ using document root <none> and password file <none>
I20260924 00:17:56.855028 18967 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:56.855096 18967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:56.855261 18967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:56.856353 18967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "0f999fc9a4ab4bbea6f84857f21faa87"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "e2ce55b27bd418214957132c4309b9ff"
server_key_iv: "b35bc7919fcbea3adfabbc2c0bc0b00c"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.856706 18967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "0f999fc9a4ab4bbea6f84857f21faa87"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "e2ce55b27bd418214957132c4309b9ff"
server_key_iv: "b35bc7919fcbea3adfabbc2c0bc0b00c"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.857972 18967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:17:56.858680 18981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.858872 18967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:17:56.858965 18967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "0f999fc9a4ab4bbea6f84857f21faa87"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "e2ce55b27bd418214957132c4309b9ff"
server_key_iv: "b35bc7919fcbea3adfabbc2c0bc0b00c"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.859040 18967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.874025 18967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.874322 18967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.874468 18967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.874686 18967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:56.875001 18967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:56.875053 18967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.875102 18967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:56.875129 18967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.880774 18967 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:38807
I20260924 00:17:56.880820 19094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:38807 every 8 connection(s)
I20260924 00:17:56.881125 18967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:17:56.886008 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.881810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:58821 (local address 127.1.44.190:33273)
0924 00:17:56.881856 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.881859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.882127 (+ 268us) server_negotiation.cc:407] Connection header received
0924 00:17:56.882546 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.882547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.882558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.882631 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:56.883332 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.883822 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.884745 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.884917 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.884966 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.885786 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:56.885788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:56.885788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:56.885792 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:56.885804 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:56.885807 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:56.885851 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:56.885932 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 00:17:56.885952 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:17:56.886210 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.881770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33273 (local address 127.1.44.130:58821)
0924 00:17:56.881955 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.881994 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.882460 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:56.882704 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:56.882707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:56.882715 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:56.883233 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.883238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.883900 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.883901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.884672 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:56.884674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:56.884984 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:56.884984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:56.885088 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:56.885645 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:17:56.885646 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:56.885684 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:17:56.885705 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:56.885863 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:56.885867 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:56.885869 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:17:56.885920 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:17:56.885995 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260924 00:17:56.886667 19095 heartbeater.cc:347] Connected to a master server at 127.1.44.190:33273
I20260924 00:17:56.886757 19095 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:56.886890 19095 heartbeater.cc:511] Master 127.1.44.190:33273 requested a full tablet report, sending...
I20260924 00:17:56.887287 18776 ts_manager.cc:194] Registered new tserver with Master: 0f999fc9a4ab4bbea6f84857f21faa87 (127.1.44.130:38807)
I20260924 00:17:56.887616 18776 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.130:58821
I20260924 00:17:56.889220 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 18967
I20260924 00:17:56.889308 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:17:56.889447 1202 external_mini_cluster.cc:1428] Setting key c8e47f9851fe320b637d3906692393d5
I20260924 00:17:56.890307 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33273
--builtin_ntp_servers=127.1.44.148:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:56.969097 19098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:56.969287 19098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:56.969308 19098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:56.970412 19098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:56.970453 19098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:56.970476 19098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:17:56.971884 19098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:56.972105 19098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:56.972375 19098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:17:56.977255 19098 server_base.cc:1034] running on GCE node
W20260924 00:17:56.977211 19104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.977210 19103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:56.977196 19106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:56.977675 19098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:56.977906 19098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:56.979054 19098 hybrid_clock.cc:648] HybridClock initialized: now 1790209076979031 us; error 36 us; skew 500 ppm
I20260924 00:17:56.980346 19098 webserver.cc:460] Webserver started at http://127.1.44.131:40119/ using document root <none> and password file <none>
I20260924 00:17:56.980575 19098 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:56.980621 19098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:56.980775 19098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:56.981791 19098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "56b1e748ebe148a6b846f8a26c7d4eb6"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "aec87c0e6150ad17920c349173ada5cf"
server_key_iv: "3325d7adb8ce2fef83d3cd8c6626dd45"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.982129 19098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "56b1e748ebe148a6b846f8a26c7d4eb6"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "aec87c0e6150ad17920c349173ada5cf"
server_key_iv: "3325d7adb8ce2fef83d3cd8c6626dd45"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.983434 19098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:17:56.984110 19112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.984287 19098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.984380 19098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "56b1e748ebe148a6b846f8a26c7d4eb6"
format_stamp: "Formatted at 2026-09-24 00:17:56 on dist-test-slave-cm7b"
server_key: "aec87c0e6150ad17920c349173ada5cf"
server_key_iv: "3325d7adb8ce2fef83d3cd8c6626dd45"
server_key_version: "encryptionkey@0"
I20260924 00:17:56.984458 19098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:56.991498 19098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:56.991791 19098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:56.991931 19098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:56.992136 19098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:56.992424 19098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:56.992475 19098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.992513 19098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:56.992542 19098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:56.998160 19098 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:45403
I20260924 00:17:56.998209 19225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:45403 every 8 connection(s)
I20260924 00:17:56.998493 19098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:17:57.003252 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.999122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:44781 (local address 127.1.44.190:33273)
0924 00:17:56.999212 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:56.999215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:56.999448 (+ 233us) server_negotiation.cc:407] Connection header received
0924 00:17:56.999898 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:56.999899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:56.999910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:56.999966 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:57.000646 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.001113 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.002014 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.002203 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.002251 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.002967 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:57.002968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:57.002968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:57.002972 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:57.002985 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:57.002988 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:57.003030 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.003107 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.003129 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:17:57.003475 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:56.999104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:33273 (local address 127.1.44.131:44781)
0924 00:17:56.999306 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:56.999357 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:56.999817 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:57.000082 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:57.000085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:57.000093 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:57.000572 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.000576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.001186 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.001187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.001947 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.001949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.002280 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.002280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.002379 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.002847 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:17:57.002847 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:57.002873 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:17:57.002894 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:57.003042 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:57.003046 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:57.003049 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:57.003099 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:17:57.003169 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":114,"threads_started":1}
I20260924 00:17:57.003954 19226 heartbeater.cc:347] Connected to a master server at 127.1.44.190:33273
I20260924 00:17:57.004040 19226 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:57.004204 19226 heartbeater.cc:511] Master 127.1.44.190:33273 requested a full tablet report, sending...
I20260924 00:17:57.004555 18776 ts_manager.cc:194] Registered new tserver with Master: 56b1e748ebe148a6b846f8a26c7d4eb6 (127.1.44.131:45403)
I20260924 00:17:57.004768 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19098
I20260924 00:17:57.004868 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:17:57.005010 18776 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.131:44781
I20260924 00:17:57.005060 1202 external_mini_cluster.cc:1428] Setting key 84e256244b7a873db8261ebb59878fe5
I20260924 00:17:57.005882 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:17:57.009691 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.007188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.1.44.190:33273)
0924 00:17:57.007249 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.007252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.007343 (+ 91us) server_negotiation.cc:407] Connection header received
0924 00:17:57.007394 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.007394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.007402 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.007446 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:57.007571 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.008008 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.008682 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.008849 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.008896 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.009507 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:57.009508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:57.009508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:57.009512 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:57.009520 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:57.009523 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:57.009561 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.009592 (+ 31us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.009606 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 00:17:57.011075 18776 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:17:57.012599 18776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:17:57.018631 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.014979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.1.44.129:41475)
0924 00:17:57.015145 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.015149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.015161 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:17:57.015223 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.015226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.015242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.015536 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:57.015592 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.016307 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.017625 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.018261 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.018344 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.018433 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.018515 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.018550 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260924 00:17:57.018623 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.014677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41475 (local address 127.0.0.1:40122)
0924 00:17:57.014793 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.014834 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:57.014899 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:57.015387 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:57.015390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:57.015392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:57.015457 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.015459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.016391 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.016392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.017539 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.017541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.018344 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.018345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.018416 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.018429 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:17:57.018502 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 00:17:57.018551 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"spinlock_wait_cycles":384,"thread_start_us":78,"threads_started":1}
I20260924 00:17:57.019011 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.014906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:38807 (local address 127.0.0.1:56994)
0924 00:17:57.015093 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.015102 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:57.015141 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:57.015589 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:57.015589 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:57.015591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:57.015624 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.015625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.016873 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.016874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.017845 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.017847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.018761 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.018762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.018826 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.018837 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:17:57.018894 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:17:57.018930 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":8,"thread_start_us":60,"threads_started":1}
I20260924 00:17:57.019063 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.015181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.1.44.130:38807)
0924 00:17:57.015388 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.015392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.015402 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:17:57.015441 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.015443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.015457 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.015530 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:57.015708 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.016780 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.017928 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.018602 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.018659 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.018728 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.018921 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.018965 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260924 00:17:57.019098 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.014787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.1.44.131:45403)
0924 00:17:57.015045 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.015051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.015151 (+ 100us) server_negotiation.cc:407] Connection header received
0924 00:17:57.015191 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.015193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.015226 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.015327 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:57.015542 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.016659 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.017829 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.018570 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.018644 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.018723 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.018971 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.019008 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":106,"threads_started":1}
I20260924 00:17:57.019217 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.014677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:45403 (local address 127.0.0.1:48834)
0924 00:17:57.015023 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.015032 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:57.015091 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:57.015402 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:57.015403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:57.015405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:57.015465 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.015467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.016746 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.016747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.017751 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.017753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.018770 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.018771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.018826 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.018835 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 00:17:57.018886 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:17:57.018930 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":5,"thread_start_us":65,"threads_started":1}
I20260924 00:17:57.019620 19029 tablet_service.cc:1467] Processing CreateTablet for tablet 361ff62a7a354ff2bcebeebbf3a1781e (DEFAULT_TABLE table=test-table [id=db184c8522b0461c9e000c1d76a7f449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:57.019620 18898 tablet_service.cc:1467] Processing CreateTablet for tablet 361ff62a7a354ff2bcebeebbf3a1781e (DEFAULT_TABLE table=test-table [id=db184c8522b0461c9e000c1d76a7f449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:57.019779 19160 tablet_service.cc:1467] Processing CreateTablet for tablet 361ff62a7a354ff2bcebeebbf3a1781e (DEFAULT_TABLE table=test-table [id=db184c8522b0461c9e000c1d76a7f449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:57.019955 19029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361ff62a7a354ff2bcebeebbf3a1781e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:57.019955 18898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361ff62a7a354ff2bcebeebbf3a1781e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:57.020012 19160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361ff62a7a354ff2bcebeebbf3a1781e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:57.022312 19240 tablet_bootstrap.cc:492] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Bootstrap starting.
I20260924 00:17:57.022869 19240 tablet_bootstrap.cc:654] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:57.023116 19242 tablet_bootstrap.cc:492] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f: Bootstrap starting.
I20260924 00:17:57.023168 19240 log.cc:824] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:57.023806 19242 tablet_bootstrap.cc:654] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:57.024060 19242 log.cc:824] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:57.024309 19241 tablet_bootstrap.cc:492] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: Bootstrap starting.
I20260924 00:17:57.024777 19242 tablet_bootstrap.cc:492] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f: No bootstrap required, opened a new log
I20260924 00:17:57.024852 19242 ts_tablet_manager.cc:1397] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:57.024938 19241 tablet_bootstrap.cc:654] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:57.025230 19241 log.cc:824] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:57.025960 19241 tablet_bootstrap.cc:492] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: No bootstrap required, opened a new log
I20260924 00:17:57.026018 19240 tablet_bootstrap.cc:492] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: No bootstrap required, opened a new log
I20260924 00:17:57.026034 19241 ts_tablet_manager.cc:1397] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:57.026072 19240 ts_tablet_manager.cc:1397] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 00:17:57.027100 19242 raft_consensus.cc:348] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.027305 19242 raft_consensus.cc:374] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:57.027351 19242 raft_consensus.cc:729] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b97f138b71194191ac2cd49eb4fc840f, State: Initialized, Role: FOLLOWER
I20260924 00:17:57.027463 19242 consensus_queue.cc:260] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.027745 18964 heartbeater.cc:503] Master 127.1.44.190:33273 was elected leader, sending a full tablet report...
I20260924 00:17:57.027729 19242 ts_tablet_manager.cc:1428] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:17:57.028631 19241 raft_consensus.cc:348] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.028792 19241 raft_consensus.cc:374] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:57.028834 19241 raft_consensus.cc:729] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f999fc9a4ab4bbea6f84857f21faa87, State: Initialized, Role: FOLLOWER
I20260924 00:17:57.028946 19241 consensus_queue.cc:260] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.029032 19240 raft_consensus.cc:348] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.029184 19240 raft_consensus.cc:374] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:57.029193 19241 ts_tablet_manager.cc:1428] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 00:17:57.029232 19240 raft_consensus.cc:729] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b1e748ebe148a6b846f8a26c7d4eb6, State: Initialized, Role: FOLLOWER
I20260924 00:17:57.029245 19095 heartbeater.cc:503] Master 127.1.44.190:33273 was elected leader, sending a full tablet report...
I20260924 00:17:57.029340 19240 consensus_queue.cc:260] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.029564 19240 ts_tablet_manager.cc:1428] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 00:17:57.029609 19226 heartbeater.cc:503] Master 127.1.44.190:33273 was elected leader, sending a full tablet report...
I20260924 00:17:57.077773 19248 raft_consensus.cc:482] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:57.077936 19248 raft_consensus.cc:504] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.078299 19248 leader_election.cc:288] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b97f138b71194191ac2cd49eb4fc840f (127.1.44.129:41475), 0f999fc9a4ab4bbea6f84857f21faa87 (127.1.44.130:38807)
I20260924 00:17:57.082231 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.078564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:53379 (local address 127.1.44.129:41475)
0924 00:17:57.078622 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.078624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.078632 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:17:57.078659 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.078660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.078674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.078747 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:57.078937 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.079709 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.081342 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.081887 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.081962 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.081969 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.082120 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.082157 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:17:57.082245 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.078749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:47135 (local address 127.1.44.130:38807)
0924 00:17:57.078802 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.078804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.078826 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:17:57.078891 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.078892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.078900 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.078951 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:57.079095 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.079797 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.081330 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.081887 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.081962 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.081969 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.082120 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.082157 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:17:57.082258 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.078616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:38807 (local address 127.1.44.131:47135)
0924 00:17:57.078772 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.078783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:57.078822 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:57.078968 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:57.078968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:57.078970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:57.079031 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.079032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.079970 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.079971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.081242 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.081245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.081969 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.081970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.082049 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.082052 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:57.082103 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:17:57.082154 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":39,"thread_start_us":102,"threads_started":1}
I20260924 00:17:57.082248 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.078479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41475 (local address 127.1.44.131:53379)
0924 00:17:57.078537 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.078549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:57.078592 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:57.078809 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:57.078810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:57.078812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:57.078877 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.078879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.079821 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.079822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.081243 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:57.081245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.081967 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.081968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:57.082042 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.082046 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:57.082103 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:17:57.082156 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":46}
I20260924 00:17:57.082666 19049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361ff62a7a354ff2bcebeebbf3a1781e" candidate_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" is_pre_election: true
I20260924 00:17:57.082854 19049 raft_consensus.cc:2454] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b1e748ebe148a6b846f8a26c7d4eb6 in term 0.
I20260924 00:17:57.082795 18918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361ff62a7a354ff2bcebeebbf3a1781e" candidate_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b97f138b71194191ac2cd49eb4fc840f" is_pre_election: true
I20260924 00:17:57.082949 18918 raft_consensus.cc:2454] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b1e748ebe148a6b846f8a26c7d4eb6 in term 0.
I20260924 00:17:57.083081 19113 leader_election.cc:302] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f999fc9a4ab4bbea6f84857f21faa87, 56b1e748ebe148a6b846f8a26c7d4eb6; no voters:
I20260924 00:17:57.083227 19248 raft_consensus.cc:2781] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:17:57.083290 19248 raft_consensus.cc:482] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:17:57.083314 19248 raft_consensus.cc:3037] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:57.083976 19248 raft_consensus.cc:504] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.084100 19248 leader_election.cc:288] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [CANDIDATE]: Term 1 election: Requested vote from peers b97f138b71194191ac2cd49eb4fc840f (127.1.44.129:41475), 0f999fc9a4ab4bbea6f84857f21faa87 (127.1.44.130:38807)
I20260924 00:17:57.084272 19049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361ff62a7a354ff2bcebeebbf3a1781e" candidate_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f999fc9a4ab4bbea6f84857f21faa87"
I20260924 00:17:57.084290 18918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361ff62a7a354ff2bcebeebbf3a1781e" candidate_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b97f138b71194191ac2cd49eb4fc840f"
I20260924 00:17:57.084364 19049 raft_consensus.cc:3037] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:57.084362 18918 raft_consensus.cc:3037] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:57.085052 18918 raft_consensus.cc:2454] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b1e748ebe148a6b846f8a26c7d4eb6 in term 1.
I20260924 00:17:57.085181 19049 raft_consensus.cc:2454] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b1e748ebe148a6b846f8a26c7d4eb6 in term 1.
I20260924 00:17:57.085202 19113 leader_election.cc:302] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b1e748ebe148a6b846f8a26c7d4eb6, b97f138b71194191ac2cd49eb4fc840f; no voters:
I20260924 00:17:57.085280 19248 raft_consensus.cc:2781] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:57.085431 19248 raft_consensus.cc:686] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 1 LEADER]: Becoming Leader. State: Replica: 56b1e748ebe148a6b846f8a26c7d4eb6, State: Running, Role: LEADER
I20260924 00:17:57.085523 19248 consensus_queue.cc:237] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } }
I20260924 00:17:57.086406 18776 catalog_manager.cc:5495] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56b1e748ebe148a6b846f8a26c7d4eb6 (127.1.44.131). New cstate: current_term: 1 leader_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56b1e748ebe148a6b846f8a26c7d4eb6" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260924 00:17:57.094705 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.092669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.1.44.131:45403)
0924 00:17:57.092720 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:57.092722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:57.092732 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:17:57.092788 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:57.092788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:57.092802 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:57.092857 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:17:57.092930 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.093388 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.094062 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:57.094242 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:57.094289 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:57.094409 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:17:57.094492 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:17:57.094548 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:57.094581 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 00:17:57.094595 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:17:57.096387 19049 raft_consensus.cc:1261] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 1 FOLLOWER]: Refusing update from remote peer 56b1e748ebe148a6b846f8a26c7d4eb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:57.096392 18918 raft_consensus.cc:1261] T 361ff62a7a354ff2bcebeebbf3a1781e P b97f138b71194191ac2cd49eb4fc840f [term 1 FOLLOWER]: Refusing update from remote peer 56b1e748ebe148a6b846f8a26c7d4eb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:57.096766 19250 consensus_queue.cc:1035] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f999fc9a4ab4bbea6f84857f21faa87" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:57.096889 19248 consensus_queue.cc:1035] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b97f138b71194191ac2cd49eb4fc840f" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:57.099184 19253 mvcc.cc:205] Tried to move back new op lower bound from 7332696379784450048 to 7332696379742941184. Current Snapshot: MvccSnapshot[applied={T|T < 7332696379784450048}]
I20260924 00:17:57.100386 19254 mvcc.cc:205] Tried to move back new op lower bound from 7332696379784450048 to 7332696379742941184. Current Snapshot: MvccSnapshot[applied={T|T < 7332696379784450048 or (T in {7332696379784450048})}]
I20260924 00:17:57.101455 18776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:57.101560 18776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:57.102279 18776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9873c8dee719415a8abfb2a427af354e: Sending DeleteTablet for 3 replicas of tablet 361ff62a7a354ff2bcebeebbf3a1781e
I20260924 00:17:57.102631 19160 tablet_service.cc:1514] Processing DeleteTablet for tablet 361ff62a7a354ff2bcebeebbf3a1781e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:57 UTC) from {username='slave'} at 127.0.0.1:48834
I20260924 00:17:57.102631 19029 tablet_service.cc:1514] Processing DeleteTablet for tablet 361ff62a7a354ff2bcebeebbf3a1781e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:57 UTC) from {username='slave'} at 127.0.0.1:56994
I20260924 00:17:57.102732 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18836
I20260924 00:17:57.102842 18898 tablet_service.cc:1514] Processing DeleteTablet for tablet 361ff62a7a354ff2bcebeebbf3a1781e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:57 UTC) from {username='slave'} at 127.0.0.1:40122
I20260924 00:17:57.107707 19263 tablet_replica.cc:330] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: stopping tablet replica
I20260924 00:17:57.107831 19263 raft_consensus.cc:2229] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:17:57.107932 19263 raft_consensus.cc:2258] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:57.108080 19264 tablet_replica.cc:330] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87: stopping tablet replica
I20260924 00:17:57.108181 19264 raft_consensus.cc:2229] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260924 00:17:57.108379 18760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:41475 (error 108)
I20260924 00:17:57.108407 19264 raft_consensus.cc:2258] T 361ff62a7a354ff2bcebeebbf3a1781e P 0f999fc9a4ab4bbea6f84857f21faa87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:57.108610 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18967
I20260924 00:17:57.108741 19263 ts_tablet_manager.cc:1905] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:57.110906 19263 ts_tablet_manager.cc:1918] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:57.110976 19263 log.cc:1196] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/361ff62a7a354ff2bcebeebbf3a1781e
I20260924 00:17:57.111253 19263 ts_tablet_manager.cc:1939] T 361ff62a7a354ff2bcebeebbf3a1781e P 56b1e748ebe148a6b846f8a26c7d4eb6: Deleting consensus metadata
I20260924 00:17:57.111837 18761 catalog_manager.cc:4841] TS 56b1e748ebe148a6b846f8a26c7d4eb6 (127.1.44.131:45403): tablet 361ff62a7a354ff2bcebeebbf3a1781e (table test-table [id=db184c8522b0461c9e000c1d76a7f449]) successfully deleted
I20260924 00:17:57.114605 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19098
I20260924 00:17:57.115274 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.114748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41475 (local address 127.0.0.1:40128)
0924 00:17:57.115161 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.115232 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394}
I20260924 00:17:57.115377 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:57.114813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:38807 (local address 127.0.0.1:57008)
0924 00:17:57.115337 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:57.115356 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.130:38807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514}
W20260924 00:17:57.116982 18760 catalog_manager.cc:4584] TS b97f138b71194191ac2cd49eb4fc840f (127.1.44.129:41475): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 361ff62a7a354ff2bcebeebbf3a1781e: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41475: connect: Connection refused (error 111)
I20260924 00:17:57.120360 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 18745
2026-09-24T00:17:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (688 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:17:57 dist-test-slave-cm7b krb5kdc[19268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:17:57 dist-test-slave-cm7b krb5kdc[19268](info): set up 2 sockets
Sep 24 00:17:57 dist-test-slave-cm7b krb5kdc[19268](info): commencing operation
krb5kdc: starting...
W20260924 00:17:58.138846 1202 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:17:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:17:58.251374 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37955
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:58.332677 19281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:58.332906 19281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:58.332942 19281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:58.334069 19281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:17:58.334148 19281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:58.334175 19281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:58.334205 19281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:17:58.334230 19281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:17:58.335673 19281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:58.335928 19281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:58.336145 19281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:58.340899 19289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:58.340899 19287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.341044 19281 server_base.cc:1034] running on GCE node
W20260924 00:17:58.340907 19286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.341462 19281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:58.341709 19281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:58.342870 19281 hybrid_clock.cc:648] HybridClock initialized: now 1790209078342856 us; error 29 us; skew 500 ppm
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:58.345484 19281 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:17:58.345939 19281 webserver.cc:460] Webserver started at http://127.1.44.190:34513/ using document root <none> and password file <none>
I20260924 00:17:58.346156 19281 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:58.346200 19281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:58.346351 19281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:58.347455 19281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "7b7197c95f934434b28d1750e2eda5c0"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "22b95753ebcd1ad4a91819149db55767"
server_key_iv: "2ddddc452fe32451bc3d0221680b1ee9"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.347817 19281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "7b7197c95f934434b28d1750e2eda5c0"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "22b95753ebcd1ad4a91819149db55767"
server_key_iv: "2ddddc452fe32451bc3d0221680b1ee9"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.349103 19281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:58.349840 19296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.350018 19281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:17:58.350109 19281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "7b7197c95f934434b28d1750e2eda5c0"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "22b95753ebcd1ad4a91819149db55767"
server_key_iv: "2ddddc452fe32451bc3d0221680b1ee9"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.350183 19281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:58.369415 19281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:58.371773 19281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:58.371922 19281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:58.376317 19281 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:37955
I20260924 00:17:58.376396 19348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:37955 every 8 connection(s)
I20260924 00:17:58.376713 19281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:17:58.377527 19349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:58.379685 19349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0: Bootstrap starting.
I20260924 00:17:58.380196 19349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:58.380417 19349 log.cc:824] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:58.381042 19349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0: No bootstrap required, opened a new log
I20260924 00:17:58.382496 19349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7197c95f934434b28d1750e2eda5c0" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37955 } }
I20260924 00:17:58.382628 19349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:58.382661 19349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7197c95f934434b28d1750e2eda5c0, State: Initialized, Role: FOLLOWER
I20260924 00:17:58.382761 19349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7197c95f934434b28d1750e2eda5c0" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37955 } }
I20260924 00:17:58.382808 19349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:17:58.382828 19349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:17:58.382854 19349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:58.383332 19349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7197c95f934434b28d1750e2eda5c0" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37955 } }
I20260924 00:17:58.383411 19349 leader_election.cc:302] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b7197c95f934434b28d1750e2eda5c0; no voters:
I20260924 00:17:58.383526 19349 leader_election.cc:288] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:17:58.383595 19352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:58.383744 19352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [term 1 LEADER]: Becoming Leader. State: Replica: 7b7197c95f934434b28d1750e2eda5c0, State: Running, Role: LEADER
I20260924 00:17:58.383762 19349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:17:58.383833 19352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7197c95f934434b28d1750e2eda5c0" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37955 } }
I20260924 00:17:58.384224 19354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b7197c95f934434b28d1750e2eda5c0. Latest consensus state: current_term: 1 leader_uuid: "7b7197c95f934434b28d1750e2eda5c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7197c95f934434b28d1750e2eda5c0" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37955 } } }
I20260924 00:17:58.384308 19354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:58.384455 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b7197c95f934434b28d1750e2eda5c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7197c95f934434b28d1750e2eda5c0" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37955 } } }
I20260924 00:17:58.384567 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0 [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:58.384537 19361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:17:58.385293 19361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:17:58.385738 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19281
I20260924 00:17:58.385871 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:17:58.386065 1202 external_mini_cluster.cc:1428] Setting key 08937d79c1e730fe8332333eb79f7d4d
I20260924 00:17:58.386930 19361 catalog_manager.cc:1347] Generated new cluster ID: 1762b700257d4ea0a05b58a8bfae4d7a
I20260924 00:17:58.386982 19361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:58.395936 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.386599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.1.44.190:37955)
0924 00:17:58.386765 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.386775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.386808 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:17:58.386919 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.386927 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.386983 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.387126 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:58.387789 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.388684 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.389558 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.390007 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.390126 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.392719 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:58.392724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:58.392724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:58.392735 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:58.394393 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.394768 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.394771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.394771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.394869 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.395075 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.395076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.395077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.395370 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:58.395531 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.395625 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.395763 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260924 00:17:58.410068 19361 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:17:58.410789 19361 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:17:58.417762 19361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0: Generated new TSK 0
I20260924 00:17:58.417935 19361 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:17:58.445833 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37955
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:58.524915 19375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:58.525107 19375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:58.525130 19375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:58.526265 19375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:58.526306 19375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:58.526331 19375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:17:58.527797 19375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:58.528014 19375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:58.528260 19375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:58.533206 19383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.533246 19375 server_base.cc:1034] running on GCE node
W20260924 00:17:58.533214 19380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:58.533339 19381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.533627 19375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:58.533876 19375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:58.535022 19375 hybrid_clock.cc:648] HybridClock initialized: now 1790209078535010 us; error 27 us; skew 500 ppm
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:58.537364 19375 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:17:58.537766 19375 webserver.cc:460] Webserver started at http://127.1.44.129:41647/ using document root <none> and password file <none>
I20260924 00:17:58.537978 19375 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:58.538045 19375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:58.538196 19375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:58.539173 19375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "565f14d22ff84299ab91c07128cc38b9"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "ae5d65925a53fd9df9290636eabf2030"
server_key_iv: "d3ecf41eaf072a59586ee7610f4d0bf2"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.539590 19375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "565f14d22ff84299ab91c07128cc38b9"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "ae5d65925a53fd9df9290636eabf2030"
server_key_iv: "d3ecf41eaf072a59586ee7610f4d0bf2"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.540930 19375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:58.541589 19390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.541767 19375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:17:58.541860 19375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "565f14d22ff84299ab91c07128cc38b9"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "ae5d65925a53fd9df9290636eabf2030"
server_key_iv: "d3ecf41eaf072a59586ee7610f4d0bf2"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.541941 19375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:58.553052 19375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:58.555012 19375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:58.555176 19375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:58.555418 19375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:58.555714 19375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:58.555765 19375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.555806 19375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:58.555835 19375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.561589 19375 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:39369
I20260924 00:17:58.561662 19503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:39369 every 8 connection(s)
I20260924 00:17:58.561964 19375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:58.570618 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.562723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:47755 (local address 127.1.44.190:37955)
0924 00:17:58.562771 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.562773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.563012 (+ 239us) server_negotiation.cc:407] Connection header received
0924 00:17:58.563599 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.563601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.563613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.563670 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:58.564573 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.565064 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.565985 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.566178 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.566233 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.568229 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:58.568230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:58.568230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:58.568236 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:58.569459 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.569811 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.569812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.569812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.569861 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.570056 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.570056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.570056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.570150 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:58.570198 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.570405 (+ 207us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.570509 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:17:58.570638 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.562706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37955 (local address 127.1.44.129:47755)
0924 00:17:58.562890 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:58.562925 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:58.563489 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:58.563744 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:58.563747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:58.564291 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:58.564481 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.564486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.565146 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.565147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.565901 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.565902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.566251 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.566252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.566343 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.566777 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:58.566784 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:58.568127 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:58.569533 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:58.569539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:58.569541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:58.569748 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:58.569918 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:58.569918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:58.569919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:58.570006 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:58.570203 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:58.570206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:58.570304 (+ 98us) client_negotiation.cc:769] Sending connection context
0924 00:17:58.570380 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:17:58.570484 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260924 00:17:58.570640 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19375
I20260924 00:17:58.570747 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:17:58.570955 1202 external_mini_cluster.cc:1428] Setting key 84774fb87079d7b7d3032c1cc0950a1a
I20260924 00:17:58.571338 19504 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37955
I20260924 00:17:58.571446 19504 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:58.571625 19504 heartbeater.cc:511] Master 127.1.44.190:37955 requested a full tablet report, sending...
I20260924 00:17:58.572173 19313 ts_manager.cc:194] Registered new tserver with Master: 565f14d22ff84299ab91c07128cc38b9 (127.1.44.129:39369)
I20260924 00:17:58.573050 19313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:47755
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:17:58.594344 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37955
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:58.672539 19509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:58.672729 19509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:58.672751 19509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:58.673820 19509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:58.673853 19509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:58.673877 19509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:17:58.675228 19509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:58.675395 19509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:58.675549 19509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:58.680265 19517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:58.680243 19514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.680295 19509 server_base.cc:1034] running on GCE node
W20260924 00:17:58.680240 19515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.680665 19509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:58.680917 19509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:58.682106 19509 hybrid_clock.cc:648] HybridClock initialized: now 1790209078682078 us; error 27 us; skew 500 ppm
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:58.684772 19509 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:17:58.685156 19509 webserver.cc:460] Webserver started at http://127.1.44.130:38201/ using document root <none> and password file <none>
I20260924 00:17:58.685369 19509 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:58.685432 19509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:58.685590 19509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:58.686568 19509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "05174380d8ae48098940d5ac8d94a871"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "f9ecec87ce7d55bd1fa5a2aa88c5fb8e"
server_key_iv: "4ea5fec8d9bd09376cf4761bbd19f55d"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.686909 19509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "05174380d8ae48098940d5ac8d94a871"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "f9ecec87ce7d55bd1fa5a2aa88c5fb8e"
server_key_iv: "4ea5fec8d9bd09376cf4761bbd19f55d"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.688234 19509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:17:58.688886 19524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.689052 19509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.689139 19509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "05174380d8ae48098940d5ac8d94a871"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "f9ecec87ce7d55bd1fa5a2aa88c5fb8e"
server_key_iv: "4ea5fec8d9bd09376cf4761bbd19f55d"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.689216 19509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:58.698823 19509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:58.700572 19509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:58.700722 19509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:58.700932 19509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:58.701202 19509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:58.701251 19509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.701282 19509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:58.701313 19509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.706939 19509 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:37351
I20260924 00:17:58.706990 19637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:37351 every 8 connection(s)
I20260924 00:17:58.707324 19509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:17:58.708448 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19509
I20260924 00:17:58.708536 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:17:58.708738 1202 external_mini_cluster.cc:1428] Setting key d3c6c6ade4577f97358f8880a2efd1a4
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
I20260924 00:17:58.717249 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.707994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:43207 (local address 127.1.44.190:37955)
0924 00:17:58.708080 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.708082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.708258 (+ 176us) server_negotiation.cc:407] Connection header received
0924 00:17:58.708897 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.708898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.708911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.708961 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:58.709920 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.710412 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.711530 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.711880 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.711955 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.714354 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:58.714355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:58.714355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:58.714361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:58.715705 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.716198 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.716199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.716199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.716245 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.716541 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.716542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.716542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.716637 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:58.716685 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.717112 (+ 427us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.717190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 00:17:58.717396 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.707964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37955 (local address 127.1.44.130:43207)
0924 00:17:58.708136 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:58.708177 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:58.708745 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:58.709036 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:58.709040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:58.709648 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:58.709822 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.709827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.710516 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.710518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.711440 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.711442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.711961 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.711962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.712058 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.712573 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:58.712587 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:58.714264 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:58.715798 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:58.715806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:58.715808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:58.716110 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:58.716328 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:58.716329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:58.716330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:58.716469 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:58.716699 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:58.716704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:58.716862 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 00:17:58.716988 (+ 126us) client_negotiation.cc:240] Negotiation successful
0924 00:17:58.717154 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260924 00:17:58.717846 19638 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37955
I20260924 00:17:58.717937 19638 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:58.718096 19638 heartbeater.cc:511] Master 127.1.44.190:37955 requested a full tablet report, sending...
I20260924 00:17:58.718457 19313 ts_manager.cc:194] Registered new tserver with Master: 05174380d8ae48098940d5ac8d94a871 (127.1.44.130:37351)
I20260924 00:17:58.718894 19313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:43207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:17:58.732242 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37955
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:17:58.810885 19643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:58.811108 19643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:58.811144 19643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:58.812280 19643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:58.812345 19643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:58.812388 19643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:17:58.813818 19643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:58.814069 19643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:58.814291 19643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:58.819156 19649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.819180 19643 server_base.cc:1034] running on GCE node
W20260924 00:17:58.819149 19648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:58.819149 19651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:58.819559 19643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:58.819792 19643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:58.820945 19643 hybrid_clock.cc:648] HybridClock initialized: now 1790209078820936 us; error 32 us; skew 500 ppm
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:17:58.823323 19643 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:17:58.823745 19643 webserver.cc:460] Webserver started at http://127.1.44.131:40711/ using document root <none> and password file <none>
I20260924 00:17:58.823963 19643 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:58.824028 19643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:58.824182 19643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:58.825178 19643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "64bbbd4f768e4fc191ee84f05fdc77ea"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "c66972e1ed61980b27f6c927161c1a00"
server_key_iv: "d3834448cde18a9ef36d6adfbe13a633"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.825534 19643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "64bbbd4f768e4fc191ee84f05fdc77ea"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "c66972e1ed61980b27f6c927161c1a00"
server_key_iv: "d3834448cde18a9ef36d6adfbe13a633"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.826814 19643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:17:58.827536 19658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.827693 19643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.827788 19643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "64bbbd4f768e4fc191ee84f05fdc77ea"
format_stamp: "Formatted at 2026-09-24 00:17:58 on dist-test-slave-cm7b"
server_key: "c66972e1ed61980b27f6c927161c1a00"
server_key_iv: "d3834448cde18a9ef36d6adfbe13a633"
server_key_version: "encryptionkey@0"
I20260924 00:17:58.827875 19643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:58.836915 19643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:58.838629 19643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:58.838785 19643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:58.838995 19643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:58.839346 19643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:58.839399 19643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.839434 19643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:58.839462 19643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:58.845132 19643 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:33561
I20260924 00:17:58.845170 19771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:33561 every 8 connection(s)
I20260924 00:17:58.845494 19643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:17:58.846447 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19643
I20260924 00:17:58.846557 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:17:58.846751 1202 external_mini_cluster.cc:1428] Setting key ec4358cbc74bb2210ddce30d3c36302a
Sep 24 00:17:58 dist-test-slave-cm7b krb5kdc[19268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209078, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:17:58.854976 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.846239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:36403 (local address 127.1.44.190:37955)
0924 00:17:58.846309 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.846311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.846489 (+ 178us) server_negotiation.cc:407] Connection header received
0924 00:17:58.847155 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.847156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.847169 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.847275 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:58.848179 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.848704 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.849758 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.849971 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.850064 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.852203 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:58.852204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:58.852204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:58.852212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:58.853588 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.853989 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.853991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.853991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.854047 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.854314 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.854315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.854315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.854433 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:58.854541 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.854808 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.854887 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 00:17:58.855154 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.846206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37955 (local address 127.1.44.131:36403)
0924 00:17:58.846388 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:58.846419 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:58.847046 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:58.847333 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:58.847337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:58.847891 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:58.848077 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.848082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.848842 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.848843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.849660 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.849662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.850072 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.850073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.850208 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.850670 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:17:58.850678 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:58.852110 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:58.853675 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:58.853682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:58.853684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:58.853900 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:58.854124 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:17:58.854125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:17:58.854125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:17:58.854226 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:17:58.854533 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:58.854539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:58.854671 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 00:17:58.854779 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 00:17:58.854930 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":125,"threads_started":1}
I20260924 00:17:58.855628 19772 heartbeater.cc:347] Connected to a master server at 127.1.44.190:37955
I20260924 00:17:58.855722 19772 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:58.855861 19772 heartbeater.cc:511] Master 127.1.44.190:37955 requested a full tablet report, sending...
I20260924 00:17:58.856190 19313 ts_manager.cc:194] Registered new tserver with Master: 64bbbd4f768e4fc191ee84f05fdc77ea (127.1.44.131:33561)
I20260924 00:17:58.856520 19313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:36403
I20260924 00:17:58.858397 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:17:58.865255 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.859692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.1.44.190:37955)
0924 00:17:58.859757 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.859758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.859885 (+ 127us) server_negotiation.cc:407] Connection header received
0924 00:17:58.859908 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.859908 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.859916 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.859959 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:58.860444 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.860886 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.861549 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.861716 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.861761 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.862976 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:58.862977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:58.862977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:17:58.862984 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:58.864237 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.864587 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.864588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.864588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.864635 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:17:58.864827 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:17:58.864827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:17:58.864827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:17:58.864936 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:58.865010 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.865112 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.865155 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 00:17:58.866636 19313 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:17:58.868184 19313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:17:58.874347 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.870240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37351 (local address 127.0.0.1:45626)
0924 00:17:58.870564 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:58.870573 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:58.870609 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:58.870802 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:58.870805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:58.870807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:58.870876 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.870899 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.871907 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.871907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.873199 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.873202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.874016 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.874017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.874105 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.874118 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:17:58.874219 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 00:17:58.874271 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":84,"threads_started":1}
I20260924 00:17:58.874357 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.870426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.1.44.130:37351)
0924 00:17:58.870618 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.870623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.870633 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:17:58.870671 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.870673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.870690 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.870776 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:58.870988 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.871807 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.873302 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.873937 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.874017 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.874077 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.874236 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.874275 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260924 00:17:58.875085 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.870249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:39369 (local address 127.0.0.1:37042)
0924 00:17:58.870666 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:58.870676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:58.870729 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:58.871121 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:58.871122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:58.871124 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:58.871188 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.871189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.872153 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.872153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.873761 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.873765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.874849 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.874850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.874911 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.874915 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:58.874974 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 00:17:58.875013 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":11,"thread_start_us":74,"threads_started":1}
I20260924 00:17:58.875108 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.870503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.1.44.129:39369)
0924 00:17:58.870913 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.870918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.870926 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:17:58.870964 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.870966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.870980 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.871049 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:58.871306 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.872052 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.873853 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.874606 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.874675 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.874724 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.874997 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.875036 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":79,"threads_started":1}
I20260924 00:17:58.875268 19572 tablet_service.cc:1467] Processing CreateTablet for tablet c860487fd6ed4c718a7505819e2f0ba5 (DEFAULT_TABLE table=test-table [id=ecb3becae7eb41c09eb21dc188c342d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:58.875619 19572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c860487fd6ed4c718a7505819e2f0ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:58.875938 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.870238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:33561 (local address 127.0.0.1:57976)
0924 00:17:58.870366 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:58.870402 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:58.870467 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:58.872385 (+ 1918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:58.872386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:58.872388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:58.872421 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.872423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.873551 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.873552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.874800 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:58.874802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.875693 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.875694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:58.875765 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.875776 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 00:17:58.875833 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 00:17:58.875872 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260924 00:17:58.875948 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:58.870482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.1.44.131:33561)
0924 00:17:58.872220 (+ 1738us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:58.872224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:58.872233 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:17:58.872277 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:58.872279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:58.872295 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:58.872369 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:58.872486 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.873460 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.874882 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:58.875627 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:58.875694 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:58.875744 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:58.875845 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 00:17:58.875877 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1706,"thread_start_us":96,"threads_started":1}
I20260924 00:17:58.876056 19438 tablet_service.cc:1467] Processing CreateTablet for tablet c860487fd6ed4c718a7505819e2f0ba5 (DEFAULT_TABLE table=test-table [id=ecb3becae7eb41c09eb21dc188c342d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:58.876363 19438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c860487fd6ed4c718a7505819e2f0ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:58.879045 19786 tablet_bootstrap.cc:492] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Bootstrap starting.
I20260924 00:17:58.879702 19786 tablet_bootstrap.cc:654] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:58.879865 19705 tablet_service.cc:1467] Processing CreateTablet for tablet c860487fd6ed4c718a7505819e2f0ba5 (DEFAULT_TABLE table=test-table [id=ecb3becae7eb41c09eb21dc188c342d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:58.880103 19786 log.cc:824] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:58.880198 19705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c860487fd6ed4c718a7505819e2f0ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:58.880726 19786 tablet_bootstrap.cc:492] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: No bootstrap required, opened a new log
I20260924 00:17:58.880785 19786 ts_tablet_manager.cc:1397] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:58.882220 19786 raft_consensus.cc:348] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:58.882458 19786 raft_consensus.cc:374] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:58.882506 19786 raft_consensus.cc:729] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05174380d8ae48098940d5ac8d94a871, State: Initialized, Role: FOLLOWER
I20260924 00:17:58.882653 19786 consensus_queue.cc:260] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:58.882687 19789 tablet_bootstrap.cc:492] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Bootstrap starting.
I20260924 00:17:58.882786 19787 tablet_bootstrap.cc:492] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: Bootstrap starting.
I20260924 00:17:58.882928 19786 ts_tablet_manager.cc:1428] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:58.883060 19638 heartbeater.cc:503] Master 127.1.44.190:37955 was elected leader, sending a full tablet report...
I20260924 00:17:58.883359 19789 tablet_bootstrap.cc:654] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:58.883448 19787 tablet_bootstrap.cc:654] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:58.883987 19789 log.cc:824] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:58.884027 19787 log.cc:824] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:58.884667 19787 tablet_bootstrap.cc:492] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: No bootstrap required, opened a new log
I20260924 00:17:58.884752 19787 ts_tablet_manager.cc:1397] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:58.886260 19787 raft_consensus.cc:348] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:58.886399 19787 raft_consensus.cc:374] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:58.886420 19787 raft_consensus.cc:729] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565f14d22ff84299ab91c07128cc38b9, State: Initialized, Role: FOLLOWER
I20260924 00:17:58.886490 19787 consensus_queue.cc:260] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:58.886718 19787 ts_tablet_manager.cc:1428] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:17:58.886735 19504 heartbeater.cc:503] Master 127.1.44.190:37955 was elected leader, sending a full tablet report...
I20260924 00:17:58.887140 19789 tablet_bootstrap.cc:492] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: No bootstrap required, opened a new log
I20260924 00:17:58.887235 19789 ts_tablet_manager.cc:1397] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260924 00:17:58.888690 19789 raft_consensus.cc:348] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:58.888834 19789 raft_consensus.cc:374] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:58.888880 19789 raft_consensus.cc:729] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bbbd4f768e4fc191ee84f05fdc77ea, State: Initialized, Role: FOLLOWER
I20260924 00:17:58.888984 19789 consensus_queue.cc:260] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:58.889221 19789 ts_tablet_manager.cc:1428] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:58.889287 19772 heartbeater.cc:503] Master 127.1.44.190:37955 was elected leader, sending a full tablet report...
W20260924 00:17:58.957818 19639 tablet.cc:2369] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:17:59.062713 19505 tablet.cc:2369] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:17:59.096091 19773 tablet.cc:2369] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:17:59.097652 19793 raft_consensus.cc:482] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:59.097754 19793 raft_consensus.cc:504] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:59.098047 19793 leader_election.cc:288] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64bbbd4f768e4fc191ee84f05fdc77ea (127.1.44.131:33561), 05174380d8ae48098940d5ac8d94a871 (127.1.44.130:37351)
I20260924 00:17:59.102193 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.098313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:54597 (local address 127.1.44.131:33561)
0924 00:17:59.098369 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.098372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.098476 (+ 104us) server_negotiation.cc:407] Connection header received
0924 00:17:59.098519 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.098520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.098530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.098614 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.098812 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.099613 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.101176 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.101770 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.101846 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.101855 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.102071 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.102123 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 00:17:59.102180 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.098246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:37351 (local address 127.1.44.129:42883)
0924 00:17:59.098385 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.098397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.098468 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.098636 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.098637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.098639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.098734 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.098736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.099705 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.099705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.101077 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.101078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.101860 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.101860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.101945 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.101954 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.102020 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.102090 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":94,"thread_start_us":44,"threads_started":1}
I20260924 00:17:59.102193 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.098354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:42883 (local address 127.1.44.130:37351)
0924 00:17:59.098404 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.098406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.098479 (+ 73us) server_negotiation.cc:407] Connection header received
0924 00:17:59.098519 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.098520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.098530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.098614 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.098813 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.099600 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.101182 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.101766 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.101846 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.101855 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.102071 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.102123 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 00:17:59.102180 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.098246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:33561 (local address 127.1.44.129:54597)
0924 00:17:59.098385 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.098397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.098468 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.098636 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.098637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.098640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.098737 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.098738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.099704 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.099705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.101071 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.101075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.101858 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.101860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.101945 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.101953 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.102020 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.102072 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":79,"thread_start_us":72,"threads_started":1}
I20260924 00:17:59.102764 19726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860487fd6ed4c718a7505819e2f0ba5" candidate_uuid: "565f14d22ff84299ab91c07128cc38b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" is_pre_election: true
I20260924 00:17:59.102805 19592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860487fd6ed4c718a7505819e2f0ba5" candidate_uuid: "565f14d22ff84299ab91c07128cc38b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05174380d8ae48098940d5ac8d94a871" is_pre_election: true
I20260924 00:17:59.102929 19726 raft_consensus.cc:2454] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565f14d22ff84299ab91c07128cc38b9 in term 0.
I20260924 00:17:59.102944 19592 raft_consensus.cc:2454] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 565f14d22ff84299ab91c07128cc38b9 in term 0.
I20260924 00:17:59.103196 19394 leader_election.cc:302] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 565f14d22ff84299ab91c07128cc38b9, 64bbbd4f768e4fc191ee84f05fdc77ea; no voters:
I20260924 00:17:59.103395 19793 raft_consensus.cc:2781] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:17:59.103444 19793 raft_consensus.cc:482] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:17:59.103461 19793 raft_consensus.cc:3037] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.104097 19793 raft_consensus.cc:504] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:59.104242 19793 leader_election.cc:288] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [CANDIDATE]: Term 1 election: Requested vote from peers 64bbbd4f768e4fc191ee84f05fdc77ea (127.1.44.131:33561), 05174380d8ae48098940d5ac8d94a871 (127.1.44.130:37351)
I20260924 00:17:59.104377 19592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860487fd6ed4c718a7505819e2f0ba5" candidate_uuid: "565f14d22ff84299ab91c07128cc38b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05174380d8ae48098940d5ac8d94a871"
I20260924 00:17:59.104447 19592 raft_consensus.cc:3037] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.104471 19726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860487fd6ed4c718a7505819e2f0ba5" candidate_uuid: "565f14d22ff84299ab91c07128cc38b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea"
I20260924 00:17:59.104552 19726 raft_consensus.cc:3037] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.105146 19592 raft_consensus.cc:2454] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565f14d22ff84299ab91c07128cc38b9 in term 1.
I20260924 00:17:59.105337 19726 raft_consensus.cc:2454] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 565f14d22ff84299ab91c07128cc38b9 in term 1.
I20260924 00:17:59.105330 19393 leader_election.cc:302] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05174380d8ae48098940d5ac8d94a871, 565f14d22ff84299ab91c07128cc38b9; no voters:
I20260924 00:17:59.105450 19793 raft_consensus.cc:2781] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:59.105634 19793 raft_consensus.cc:686] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 1 LEADER]: Becoming Leader. State: Replica: 565f14d22ff84299ab91c07128cc38b9, State: Running, Role: LEADER
I20260924 00:17:59.105738 19793 consensus_queue.cc:237] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } }
I20260924 00:17:59.106757 19312 catalog_manager.cc:5495] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 565f14d22ff84299ab91c07128cc38b9 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "565f14d22ff84299ab91c07128cc38b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "565f14d22ff84299ab91c07128cc38b9" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 39369 } health_report { overall_health: HEALTHY } } }
I20260924 00:17:59.160758 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.158199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.1.44.129:39369)
0924 00:17:59.158265 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.158268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.158277 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:17:59.158340 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.158341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.158364 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.158454 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:17:59.158595 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.159227 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.159999 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.160209 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.160273 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.160421 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:17:59.160505 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:17:59.160577 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.160626 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.160635 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:17:59.162717 19592 raft_consensus.cc:1261] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 1 FOLLOWER]: Refusing update from remote peer 565f14d22ff84299ab91c07128cc38b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:59.162730 19726 raft_consensus.cc:1261] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 1 FOLLOWER]: Refusing update from remote peer 565f14d22ff84299ab91c07128cc38b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:59.163120 19797 consensus_queue.cc:1035] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05174380d8ae48098940d5ac8d94a871" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 37351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:59.163297 19793 consensus_queue.cc:1035] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bbbd4f768e4fc191ee84f05fdc77ea" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 33561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:59.165458 19800 mvcc.cc:205] Tried to move back new op lower bound from 7332696388247412736 to 7332696388017762304. Current Snapshot: MvccSnapshot[applied={T|T < 7332696388247412736}]
I20260924 00:17:59.165628 19802 mvcc.cc:205] Tried to move back new op lower bound from 7332696388247412736 to 7332696388017762304. Current Snapshot: MvccSnapshot[applied={T|T < 7332696388247412736}]
I20260924 00:17:59.167893 19313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:59.167986 19313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:59.168731 19313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b7197c95f934434b28d1750e2eda5c0: Sending DeleteTablet for 3 replicas of tablet c860487fd6ed4c718a7505819e2f0ba5
I20260924 00:17:59.169023 19705 tablet_service.cc:1514] Processing DeleteTablet for tablet c860487fd6ed4c718a7505819e2f0ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:59 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:57976
I20260924 00:17:59.169051 19438 tablet_service.cc:1514] Processing DeleteTablet for tablet c860487fd6ed4c718a7505819e2f0ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:59 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:37042
I20260924 00:17:59.169159 19572 tablet_service.cc:1514] Processing DeleteTablet for tablet c860487fd6ed4c718a7505819e2f0ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:59 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:45626
I20260924 00:17:59.169389 19812 tablet_replica.cc:330] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: stopping tablet replica
I20260924 00:17:59.169471 19811 tablet_replica.cc:330] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9: stopping tablet replica
I20260924 00:17:59.169486 19812 raft_consensus.cc:2229] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:59.169581 19812 raft_consensus.cc:2258] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:59.169577 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19375
I20260924 00:17:59.169607 19811 raft_consensus.cc:2229] T c860487fd6ed4c718a7505819e2f0ba5 P 565f14d22ff84299ab91c07128cc38b9 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:17:59.170199 19810 tablet_replica.cc:330] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: stopping tablet replica
I20260924 00:17:59.170321 19810 raft_consensus.cc:2229] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:59.170419 19810 raft_consensus.cc:2258] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:59.171473 19812 ts_tablet_manager.cc:1905] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:59.171859 19810 ts_tablet_manager.cc:1905] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:59.173352 19810 ts_tablet_manager.cc:1918] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:59.173444 19810 log.cc:1196] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/c860487fd6ed4c718a7505819e2f0ba5
I20260924 00:17:59.173496 19812 ts_tablet_manager.cc:1918] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:59.173568 19812 log.cc:1196] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/c860487fd6ed4c718a7505819e2f0ba5
I20260924 00:17:59.173746 19810 ts_tablet_manager.cc:1939] T c860487fd6ed4c718a7505819e2f0ba5 P 64bbbd4f768e4fc191ee84f05fdc77ea: Deleting consensus metadata
I20260924 00:17:59.173844 19812 ts_tablet_manager.cc:1939] T c860487fd6ed4c718a7505819e2f0ba5 P 05174380d8ae48098940d5ac8d94a871: Deleting consensus metadata
I20260924 00:17:59.174203 19300 catalog_manager.cc:4841] TS 64bbbd4f768e4fc191ee84f05fdc77ea (127.1.44.131:33561): tablet c860487fd6ed4c718a7505819e2f0ba5 (table test-table [id=ecb3becae7eb41c09eb21dc188c342d5]) successfully deleted
I20260924 00:17:59.174409 19299 catalog_manager.cc:4841] TS 05174380d8ae48098940d5ac8d94a871 (127.1.44.130:37351): tablet c860487fd6ed4c718a7505819e2f0ba5 (table test-table [id=ecb3becae7eb41c09eb21dc188c342d5]) successfully deleted
W20260924 00:17:59.176342 19298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:39369 (error 108)
I20260924 00:17:59.176599 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19509
I20260924 00:17:59.177165 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.176689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:39369 (local address 127.0.0.1:37064)
0924 00:17:59.177077 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.177133 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:39369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368}
W20260924 00:17:59.177289 19298 catalog_manager.cc:4584] TS 565f14d22ff84299ab91c07128cc38b9 (127.1.44.129:39369): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c860487fd6ed4c718a7505819e2f0ba5: Network error: Client connection negotiation failed: client connection to 127.1.44.129:39369: connect: Connection refused (error 111)
I20260924 00:17:59.182049 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19643
I20260924 00:17:59.186406 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19281
2026-09-24T00:17:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2065 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T00:17:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:17:59Z Disabled control of system clock
I20260924 00:17:59.212879 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34273
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:59.290454 19817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:59.290879 19817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:59.290921 19817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:59.292057 19817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:17:59.292094 19817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:59.292107 19817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:59.292120 19817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:17:59.292132 19817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:17:59.293386 19817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:59.293561 19817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:59.293767 19817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:59.298329 19823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:59.298355 19822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.298377 19817 server_base.cc:1034] running on GCE node
W20260924 00:17:59.298345 19825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.298846 19817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:59.299118 19817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:59.300312 19817 hybrid_clock.cc:648] HybridClock initialized: now 1790209079300304 us; error 34 us; skew 500 ppm
I20260924 00:17:59.301528 19817 webserver.cc:460] Webserver started at http://127.1.44.190:42359/ using document root <none> and password file <none>
I20260924 00:17:59.301764 19817 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:59.301834 19817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:59.301985 19817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:59.302995 19817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "d39fbd88bcb2470aa8c416d52ec5a60b"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "abdf954010538ee62065043edb704927"
server_key_iv: "2867f219199e8948bfd461fecfc6bd7b"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.303385 19817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "d39fbd88bcb2470aa8c416d52ec5a60b"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "abdf954010538ee62065043edb704927"
server_key_iv: "2867f219199e8948bfd461fecfc6bd7b"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.304649 19817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:59.305441 19831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.305613 19817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:17:59.305712 19817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "d39fbd88bcb2470aa8c416d52ec5a60b"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "abdf954010538ee62065043edb704927"
server_key_iv: "2867f219199e8948bfd461fecfc6bd7b"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.305800 19817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:59.316958 19817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:59.317245 19817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:59.317396 19817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:59.321288 19817 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34273
I20260924 00:17:59.321319 19883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34273 every 8 connection(s)
I20260924 00:17:59.321650 19817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:17:59.322599 19884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:59.324784 19884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b: Bootstrap starting.
I20260924 00:17:59.325301 19884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:59.325529 19884 log.cc:824] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:59.326130 19884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b: No bootstrap required, opened a new log
I20260924 00:17:59.326637 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19817
I20260924 00:17:59.326761 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:17:59.326951 1202 external_mini_cluster.cc:1428] Setting key 81f5bf6a3a79a4cc0a4f2e14f15a630d
I20260924 00:17:59.328320 19884 raft_consensus.cc:348] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34273 } }
I20260924 00:17:59.328482 19884 raft_consensus.cc:374] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:59.328517 19884 raft_consensus.cc:729] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39fbd88bcb2470aa8c416d52ec5a60b, State: Initialized, Role: FOLLOWER
I20260924 00:17:59.328650 19884 consensus_queue.cc:260] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34273 } }
I20260924 00:17:59.328745 19884 raft_consensus.cc:388] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:17:59.328778 19884 raft_consensus.cc:482] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:17:59.328824 19884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.329339 19884 raft_consensus.cc:504] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34273 } }
I20260924 00:17:59.329419 19884 leader_election.cc:302] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d39fbd88bcb2470aa8c416d52ec5a60b; no voters:
I20260924 00:17:59.329594 19884 leader_election.cc:288] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:17:59.329622 19889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:59.329737 19889 raft_consensus.cc:686] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [term 1 LEADER]: Becoming Leader. State: Replica: d39fbd88bcb2470aa8c416d52ec5a60b, State: Running, Role: LEADER
I20260924 00:17:59.329846 19884 sys_catalog.cc:564] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:17:59.329824 19889 consensus_queue.cc:237] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34273 } }
I20260924 00:17:59.330358 19891 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39fbd88bcb2470aa8c416d52ec5a60b. Latest consensus state: current_term: 1 leader_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34273 } } }
I20260924 00:17:59.330597 19891 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:59.330525 19890 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d39fbd88bcb2470aa8c416d52ec5a60b" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34273 } } }
I20260924 00:17:59.330870 19890 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b [sys.catalog]: This master's current role is: LEADER
I20260924 00:17:59.331061 19898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:17:59.331641 19898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:17:59.333989 19898 catalog_manager.cc:1347] Generated new cluster ID: 675324f3dd2c4dc3af4018ecc0219d8d
I20260924 00:17:59.334045 19898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:17:59.334796 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.327568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.1.44.190:34273)
0924 00:17:59.327777 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.327784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.327807 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:17:59.328300 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.328306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.328361 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.328473 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:59.328757 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.330451 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.332688 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.333201 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.333352 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.334130 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:59.334133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:59.334133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:59.334317 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:59.334349 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:59.334360 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:59.334441 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.334514 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.334593 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260924 00:17:59.351370 19898 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:17:59.351907 19898 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:17:59.356639 19898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b: Generated new TSK 0
I20260924 00:17:59.356832 19898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:17:59.364073 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34273
--builtin_ntp_servers=127.1.44.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:59.450440 19908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:59.450655 19908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:59.450677 19908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:59.452108 19908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:59.452163 19908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:59.452190 19908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:17:59.453650 19908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:59.453922 19908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:59.454231 19908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:59.459103 19913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:59.459177 19914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:59.459116 19916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.459573 19908 server_base.cc:1034] running on GCE node
I20260924 00:17:59.459779 19908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:59.460027 19908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:59.461181 19908 hybrid_clock.cc:648] HybridClock initialized: now 1790209079461154 us; error 43 us; skew 500 ppm
I20260924 00:17:59.462574 19908 webserver.cc:460] Webserver started at http://127.1.44.129:44721/ using document root <none> and password file <none>
I20260924 00:17:59.462860 19908 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:59.462929 19908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:59.463096 19908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:59.464454 19908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "0baac095ad814ef8831ded62c1604f35"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "399e716a2fd7f92c72d468ca8c2fadcf"
server_key_iv: "238c6f13023a73d31b5154ea60d0c3a5"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.464931 19908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "0baac095ad814ef8831ded62c1604f35"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "399e716a2fd7f92c72d468ca8c2fadcf"
server_key_iv: "238c6f13023a73d31b5154ea60d0c3a5"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.466506 19908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:59.467450 19922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.467684 19908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:59.467795 19908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "0baac095ad814ef8831ded62c1604f35"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "399e716a2fd7f92c72d468ca8c2fadcf"
server_key_iv: "238c6f13023a73d31b5154ea60d0c3a5"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.467883 19908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:59.482040 19908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:59.482362 19908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:59.482522 19908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:59.482757 19908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:59.483094 19908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:59.483150 19908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.483186 19908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:59.483276 19908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.488963 19908 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:35951
I20260924 00:17:59.489009 20035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:35951 every 8 connection(s)
I20260924 00:17:59.489337 19908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:17:59.494614 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.490077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:33987 (local address 127.1.44.190:34273)
0924 00:17:59.490150 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.490153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.490435 (+ 282us) server_negotiation.cc:407] Connection header received
0924 00:17:59.490900 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.490902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.490911 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.490972 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:59.491723 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.492346 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.493287 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.493478 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.493529 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.494357 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:59.494358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:59.494359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:59.494363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:59.494382 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:59.494387 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:59.494440 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.494522 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.494553 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:17:59.494813 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.490075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34273 (local address 127.1.44.129:33987)
0924 00:17:59.490289 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.490324 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.490793 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.491055 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.491058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.491067 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:59.491641 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.491647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.492425 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.492426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.493214 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.493216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.493550 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.493550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.493656 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.494199 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:17:59.494199 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:59.494234 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:17:59.494256 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:59.494453 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:59.494457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:59.494459 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.494510 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.494587 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260924 00:17:59.495318 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 19908
I20260924 00:17:59.495347 20036 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34273
I20260924 00:17:59.495427 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:17:59.495434 20036 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:59.495625 1202 external_mini_cluster.cc:1428] Setting key 13b45b4005fdd30658fe42e0a60587e5
I20260924 00:17:59.495623 20036 heartbeater.cc:511] Master 127.1.44.190:34273 requested a full tablet report, sending...
I20260924 00:17:59.496169 19847 ts_manager.cc:194] Registered new tserver with Master: 0baac095ad814ef8831ded62c1604f35 (127.1.44.129:35951)
I20260924 00:17:59.496517 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34273
--builtin_ntp_servers=127.1.44.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 00:17:59.496883 19847 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.129:33987
W20260924 00:17:59.576445 20039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:59.576635 20039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:59.576656 20039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:59.577759 20039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:59.577802 20039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:59.577852 20039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:17:59.579141 20039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:59.579414 20039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:59.579690 20039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:59.584442 20044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:59.584466 20045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.584504 20039 server_base.cc:1034] running on GCE node
W20260924 00:17:59.584502 20047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.584807 20039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:59.585004 20039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:59.586138 20039 hybrid_clock.cc:648] HybridClock initialized: now 1790209079586121 us; error 34 us; skew 500 ppm
I20260924 00:17:59.587327 20039 webserver.cc:460] Webserver started at http://127.1.44.130:46749/ using document root <none> and password file <none>
I20260924 00:17:59.587555 20039 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:59.587599 20039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:59.587744 20039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:59.588752 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "c6dcda42e1af4f87a5b785d7682a0214"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "0e498685baa3f4f45b23db1b9fb37f2e"
server_key_iv: "5f663ac8fb6f394de73d3d24d37cf74c"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.589105 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "c6dcda42e1af4f87a5b785d7682a0214"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "0e498685baa3f4f45b23db1b9fb37f2e"
server_key_iv: "5f663ac8fb6f394de73d3d24d37cf74c"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.590346 20039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:17:59.591076 20053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.591293 20039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:17:59.591390 20039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "c6dcda42e1af4f87a5b785d7682a0214"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "0e498685baa3f4f45b23db1b9fb37f2e"
server_key_iv: "5f663ac8fb6f394de73d3d24d37cf74c"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.591462 20039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:59.625768 20039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:59.626102 20039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:59.626279 20039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:59.626550 20039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:59.626922 20039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:59.626978 20039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.627019 20039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:59.627063 20039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.633251 20039 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:33323
I20260924 00:17:59.633293 20166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:33323 every 8 connection(s)
I20260924 00:17:59.633648 20039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:17:59.638510 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.634364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:49605 (local address 127.1.44.190:34273)
0924 00:17:59.634411 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.634413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.634655 (+ 242us) server_negotiation.cc:407] Connection header received
0924 00:17:59.635126 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.635127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.635139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.635224 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:59.635894 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.636373 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.637340 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.637518 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.637566 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.638282 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:59.638283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:59.638284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:59.638287 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:59.638301 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:59.638304 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:59.638346 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.638431 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.638452 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:17:59.638746 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.634357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34273 (local address 127.1.44.130:49605)
0924 00:17:59.634527 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.634564 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.635044 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.635314 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.635317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.635325 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:59.635817 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.635821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.636470 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.636471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.637261 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.637263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.637595 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.637596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.637701 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.638152 (+ 451us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:17:59.638153 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:59.638179 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:17:59.638200 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:59.638359 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:59.638363 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:59.638365 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.638416 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.638492 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260924 00:17:59.639353 20167 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34273
I20260924 00:17:59.639448 20167 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:59.639586 20167 heartbeater.cc:511] Master 127.1.44.190:34273 requested a full tablet report, sending...
I20260924 00:17:59.639941 19847 ts_manager.cc:194] Registered new tserver with Master: c6dcda42e1af4f87a5b785d7682a0214 (127.1.44.130:33323)
I20260924 00:17:59.640276 19847 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.130:49605
I20260924 00:17:59.641461 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20039
I20260924 00:17:59.641537 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:17:59.641742 1202 external_mini_cluster.cc:1428] Setting key 2463acaf9089dede7109f131b5995504
I20260924 00:17:59.642727 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34273
--builtin_ntp_servers=127.1.44.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:17:59.732105 20170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:17:59.732316 20170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:17:59.732338 20170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:17:59.733451 20170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:17:59.733485 20170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:17:59.733510 20170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:17:59.734758 20170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:17:59.734947 20170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:17:59.735275 20170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:17:59.740361 20175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.740422 20170 server_base.cc:1034] running on GCE node
W20260924 00:17:59.740361 20176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:17:59.740387 20178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:17:59.740804 20170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:17:59.741057 20170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:17:59.742219 20170 hybrid_clock.cc:648] HybridClock initialized: now 1790209079742215 us; error 34 us; skew 500 ppm
I20260924 00:17:59.743474 20170 webserver.cc:460] Webserver started at http://127.1.44.131:39729/ using document root <none> and password file <none>
I20260924 00:17:59.743728 20170 fs_manager.cc:359] Metadata directory not provided
I20260924 00:17:59.743796 20170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:17:59.743943 20170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:17:59.744969 20170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "e96b70a5216a4f9eb5eb8da0a429112c"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "b1c1664812067001779b4f8eff437866"
server_key_iv: "f41dc2c911ad94e01608981dc96a47d5"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.745319 20170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "e96b70a5216a4f9eb5eb8da0a429112c"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "b1c1664812067001779b4f8eff437866"
server_key_iv: "f41dc2c911ad94e01608981dc96a47d5"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.746569 20170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:17:59.747352 20184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.747529 20170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:17:59.747624 20170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "e96b70a5216a4f9eb5eb8da0a429112c"
format_stamp: "Formatted at 2026-09-24 00:17:59 on dist-test-slave-cm7b"
server_key: "b1c1664812067001779b4f8eff437866"
server_key_iv: "f41dc2c911ad94e01608981dc96a47d5"
server_key_version: "encryptionkey@0"
I20260924 00:17:59.747715 20170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:17:59.771355 20170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:17:59.771656 20170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:17:59.771814 20170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:17:59.772042 20170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:17:59.772370 20170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:17:59.772423 20170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.772457 20170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:17:59.772485 20170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:17:59.778116 20170 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:34207
I20260924 00:17:59.778255 20297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:34207 every 8 connection(s)
I20260924 00:17:59.778538 20170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:17:59.783478 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.779399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:35323 (local address 127.1.44.190:34273)
0924 00:17:59.779478 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.779480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.779692 (+ 212us) server_negotiation.cc:407] Connection header received
0924 00:17:59.780149 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.780150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.780160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.780215 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:59.780838 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.781334 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.782254 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.782427 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.782476 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.783235 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:59.783236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:59.783236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:59.783242 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:59.783257 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:59.783262 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:59.783319 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.783398 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.783419 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 00:17:59.783671 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.779370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34273 (local address 127.1.44.131:35323)
0924 00:17:59.779569 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.779602 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.780070 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.780269 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.780272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.780280 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:17:59.780773 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.780776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.781406 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.781407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.782180 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.782181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.782498 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.782499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.782603 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.783078 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:17:59.783079 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:17:59.783105 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:17:59.783126 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:17:59.783328 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:17:59.783332 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:17:59.783334 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.783389 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.783463 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260924 00:17:59.784246 20298 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34273
I20260924 00:17:59.784322 20298 heartbeater.cc:464] Registering TS with master...
I20260924 00:17:59.784451 20298 heartbeater.cc:511] Master 127.1.44.190:34273 requested a full tablet report, sending...
I20260924 00:17:59.784765 19847 ts_manager.cc:194] Registered new tserver with Master: e96b70a5216a4f9eb5eb8da0a429112c (127.1.44.131:34207)
I20260924 00:17:59.785130 19847 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.131:35323
I20260924 00:17:59.788223 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20170
I20260924 00:17:59.788328 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:17:59.788525 1202 external_mini_cluster.cc:1428] Setting key 9beb4c62382c5a2b5db165a4d569524c
I20260924 00:17:59.789347 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:17:59.793463 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.790631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.1.44.190:34273)
0924 00:17:59.790700 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.790702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.790778 (+ 76us) server_negotiation.cc:407] Connection header received
0924 00:17:59.790813 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.790814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.790821 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.790865 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:17:59.790980 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.791488 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.792271 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.792474 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.792542 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.793225 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:17:59.793226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:17:59.793226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:17:59.793230 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:17:59.793243 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:17:59.793247 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:17:59.793320 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.793403 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.793413 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 00:17:59.794886 19847 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:17:59.796435 19847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:17:59.802624 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.798669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.1.44.131:34207)
0924 00:17:59.799258 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.799262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.799270 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:17:59.799302 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.799303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.799318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.799391 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.799570 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.800250 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.801717 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.802239 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.802328 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.802416 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.802502 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.802538 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":88,"threads_started":1}
I20260924 00:17:59.802613 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.798442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34207 (local address 127.0.0.1:60016)
0924 00:17:59.798843 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.798851 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.798888 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.799447 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.799448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.799450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.799495 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.799497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.800337 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.800338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.801617 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.801619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.802330 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.802331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.802403 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.802416 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.802489 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.802539 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260924 00:17:59.802755 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.798432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:33323 (local address 127.0.0.1:53698)
0924 00:17:59.798614 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.798649 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.798733 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.799136 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.799137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.799139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.799168 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.799169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.800520 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.800521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.801617 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.801619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.802555 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.802556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.802612 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.802614 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.802658 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.802691 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":30,"thread_start_us":76,"threads_started":1}
I20260924 00:17:59.802809 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.798601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51928 (local address 127.1.44.129:35951)
0924 00:17:59.798746 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.798752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.798889 (+ 137us) server_negotiation.cc:407] Connection header received
0924 00:17:59.798956 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.798958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.798975 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.799066 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.799442 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.800518 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.801521 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.802266 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.802339 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.802431 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.802671 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.802725 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260924 00:17:59.803052 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.798783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.1.44.130:33323)
0924 00:17:59.798987 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.798991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.799002 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:17:59.799037 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.799039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.799054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.799127 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.799289 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.800432 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.801705 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.802389 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.802450 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.802525 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.802918 (+ 393us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.802970 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":101,"threads_started":1}
I20260924 00:17:59.803179 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.798432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:35951 (local address 127.0.0.1:51928)
0924 00:17:59.798614 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.798817 (+ 203us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.798878 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.799073 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.799075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.799077 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.799163 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.799164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.800611 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.800612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.801447 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.801449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.802468 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.802470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.802532 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.802536 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.802585 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.802624 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"spinlock_wait_cycles":768,"thread_start_us":88,"threads_started":1}
I20260924 00:17:59.803570 20232 tablet_service.cc:1467] Processing CreateTablet for tablet f2da981363084f1f9a5c088e7eca6283 (DEFAULT_TABLE table=test-table [id=311cdf15e67940b097dbc1517b3e8e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:59.803834 19970 tablet_service.cc:1467] Processing CreateTablet for tablet f2da981363084f1f9a5c088e7eca6283 (DEFAULT_TABLE table=test-table [id=311cdf15e67940b097dbc1517b3e8e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:59.803956 20232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2da981363084f1f9a5c088e7eca6283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:59.804157 19970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2da981363084f1f9a5c088e7eca6283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:59.806679 20313 tablet_bootstrap.cc:492] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: Bootstrap starting.
I20260924 00:17:59.806805 20312 tablet_bootstrap.cc:492] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: Bootstrap starting.
I20260924 00:17:59.807078 20101 tablet_service.cc:1467] Processing CreateTablet for tablet f2da981363084f1f9a5c088e7eca6283 (DEFAULT_TABLE table=test-table [id=311cdf15e67940b097dbc1517b3e8e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:17:59.807395 20101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2da981363084f1f9a5c088e7eca6283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:17:59.807411 20313 tablet_bootstrap.cc:654] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:59.807411 20312 tablet_bootstrap.cc:654] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:59.807731 20313 log.cc:824] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:59.807739 20312 log.cc:824] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:59.808419 20312 tablet_bootstrap.cc:492] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: No bootstrap required, opened a new log
I20260924 00:17:59.808470 20312 ts_tablet_manager.cc:1397] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:17:59.810425 20316 tablet_bootstrap.cc:492] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Bootstrap starting.
I20260924 00:17:59.810570 20312 raft_consensus.cc:348] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.810746 20312 raft_consensus.cc:374] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:59.810787 20312 raft_consensus.cc:729] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96b70a5216a4f9eb5eb8da0a429112c, State: Initialized, Role: FOLLOWER
I20260924 00:17:59.810907 20312 consensus_queue.cc:260] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.811087 20316 tablet_bootstrap.cc:654] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Neither blocks nor log segments found. Creating new log.
I20260924 00:17:59.811156 20312 ts_tablet_manager.cc:1428] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:17:59.811305 20298 heartbeater.cc:503] Master 127.1.44.190:34273 was elected leader, sending a full tablet report...
I20260924 00:17:59.811441 20316 log.cc:824] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Log is configured to *not* fsync() on all Append() calls
I20260924 00:17:59.811827 20313 tablet_bootstrap.cc:492] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: No bootstrap required, opened a new log
I20260924 00:17:59.811921 20313 ts_tablet_manager.cc:1397] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 00:17:59.812181 20316 tablet_bootstrap.cc:492] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: No bootstrap required, opened a new log
I20260924 00:17:59.812249 20316 ts_tablet_manager.cc:1397] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 00:17:59.814532 20313 raft_consensus.cc:348] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.814698 20313 raft_consensus.cc:374] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:59.814742 20313 raft_consensus.cc:729] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0baac095ad814ef8831ded62c1604f35, State: Initialized, Role: FOLLOWER
I20260924 00:17:59.814706 20316 raft_consensus.cc:348] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.814845 20316 raft_consensus.cc:374] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:17:59.814886 20316 raft_consensus.cc:729] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6dcda42e1af4f87a5b785d7682a0214, State: Initialized, Role: FOLLOWER
I20260924 00:17:59.814864 20313 consensus_queue.cc:260] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.814988 20316 consensus_queue.cc:260] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.815135 20313 ts_tablet_manager.cc:1428] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:17:59.815178 20036 heartbeater.cc:503] Master 127.1.44.190:34273 was elected leader, sending a full tablet report...
I20260924 00:17:59.815253 20316 ts_tablet_manager.cc:1428] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 00:17:59.815354 20167 heartbeater.cc:503] Master 127.1.44.190:34273 was elected leader, sending a full tablet report...
I20260924 00:17:59.859241 20319 raft_consensus.cc:482] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:17:59.859362 20319 raft_consensus.cc:504] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.859740 20319 leader_election.cc:288] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6dcda42e1af4f87a5b785d7682a0214 (127.1.44.130:33323), e96b70a5216a4f9eb5eb8da0a429112c (127.1.44.131:34207)
I20260924 00:17:59.862551 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.860013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:53691 (local address 127.1.44.131:34207)
0924 00:17:59.860055 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.860057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.860065 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:17:59.860154 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.860155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.860164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.860223 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.860394 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.860870 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.861830 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.862194 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.862287 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.862295 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.862450 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.862491 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:17:59.862551 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.859945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34207 (local address 127.1.44.129:53691)
0924 00:17:59.860014 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.860027 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.860081 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.860256 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.860257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.860259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.860325 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.860327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.860961 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.860961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.861740 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.861742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.862288 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.862289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.862369 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.862372 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.862432 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.862474 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":26}
I20260924 00:17:59.862684 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.859956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:33323 (local address 127.1.44.129:41627)
0924 00:17:59.860117 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.860127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:17:59.860209 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:17:59.860360 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:17:59.860361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:17:59.860363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:17:59.860433 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.860434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.861063 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.861064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.861853 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:17:59.861855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.862401 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.862402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:17:59.862489 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.862492 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:17:59.862558 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 00:17:59.862602 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":34,"thread_start_us":84,"threads_started":1}
I20260924 00:17:59.862671 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.860042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:41627 (local address 127.1.44.130:33323)
0924 00:17:59.860103 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.860105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.860199 (+ 94us) server_negotiation.cc:407] Connection header received
0924 00:17:59.860276 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.860277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.860295 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.860359 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:17:59.860518 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.860976 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.861947 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.862313 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.862399 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.862407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.862565 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.862602 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 00:17:59.862907 20252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2da981363084f1f9a5c088e7eca6283" candidate_uuid: "0baac095ad814ef8831ded62c1604f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" is_pre_election: true
I20260924 00:17:59.863065 20252 raft_consensus.cc:2454] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0baac095ad814ef8831ded62c1604f35 in term 0.
I20260924 00:17:59.863232 20121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2da981363084f1f9a5c088e7eca6283" candidate_uuid: "0baac095ad814ef8831ded62c1604f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6dcda42e1af4f87a5b785d7682a0214" is_pre_election: true
I20260924 00:17:59.863314 19924 leader_election.cc:302] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0baac095ad814ef8831ded62c1604f35, e96b70a5216a4f9eb5eb8da0a429112c; no voters:
I20260924 00:17:59.863381 20121 raft_consensus.cc:2454] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0baac095ad814ef8831ded62c1604f35 in term 0.
I20260924 00:17:59.863435 20319 raft_consensus.cc:2781] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:17:59.863516 20319 raft_consensus.cc:482] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:17:59.863549 20319 raft_consensus.cc:3037] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.864235 20319 raft_consensus.cc:504] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.864380 20319 leader_election.cc:288] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [CANDIDATE]: Term 1 election: Requested vote from peers c6dcda42e1af4f87a5b785d7682a0214 (127.1.44.130:33323), e96b70a5216a4f9eb5eb8da0a429112c (127.1.44.131:34207)
I20260924 00:17:59.864511 20252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2da981363084f1f9a5c088e7eca6283" candidate_uuid: "0baac095ad814ef8831ded62c1604f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e96b70a5216a4f9eb5eb8da0a429112c"
I20260924 00:17:59.864543 20121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2da981363084f1f9a5c088e7eca6283" candidate_uuid: "0baac095ad814ef8831ded62c1604f35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6dcda42e1af4f87a5b785d7682a0214"
I20260924 00:17:59.864591 20252 raft_consensus.cc:3037] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.864650 20121 raft_consensus.cc:3037] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:17:59.865326 20252 raft_consensus.cc:2454] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0baac095ad814ef8831ded62c1604f35 in term 1.
I20260924 00:17:59.865406 20121 raft_consensus.cc:2454] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0baac095ad814ef8831ded62c1604f35 in term 1.
I20260924 00:17:59.865485 19924 leader_election.cc:302] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0baac095ad814ef8831ded62c1604f35, e96b70a5216a4f9eb5eb8da0a429112c; no voters:
I20260924 00:17:59.865588 20319 raft_consensus.cc:2781] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:17:59.865738 20319 raft_consensus.cc:686] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 1 LEADER]: Becoming Leader. State: Replica: 0baac095ad814ef8831ded62c1604f35, State: Running, Role: LEADER
I20260924 00:17:59.865849 20319 consensus_queue.cc:237] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } }
I20260924 00:17:59.866676 19847 catalog_manager.cc:5495] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0baac095ad814ef8831ded62c1604f35 (127.1.44.129). New cstate: current_term: 1 leader_uuid: "0baac095ad814ef8831ded62c1604f35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0baac095ad814ef8831ded62c1604f35" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 35951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 } health_report { overall_health: UNKNOWN } } }
I20260924 00:17:59.875255 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.873155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.1.44.129:35951)
0924 00:17:59.873216 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 00:17:59.873218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:17:59.873225 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:17:59.873299 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:17:59.873299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:17:59.873315 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:17:59.873372 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:17:59.873481 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.873911 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.874586 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:17:59.874764 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:17:59.874813 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:17:59.874937 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:17:59.875012 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:17:59.875077 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 00:17:59.875119 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 00:17:59.875127 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:17:59.876943 20252 raft_consensus.cc:1261] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 1 FOLLOWER]: Refusing update from remote peer 0baac095ad814ef8831ded62c1604f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:59.876943 20121 raft_consensus.cc:1261] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 1 FOLLOWER]: Refusing update from remote peer 0baac095ad814ef8831ded62c1604f35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:17:59.877349 20322 consensus_queue.cc:1035] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6dcda42e1af4f87a5b785d7682a0214" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 33323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:59.877482 20319 consensus_queue.cc:1035] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e96b70a5216a4f9eb5eb8da0a429112c" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:17:59.880523 20327 mvcc.cc:205] Tried to move back new op lower bound from 7332696391173136384 to 7332696391130976256. Current Snapshot: MvccSnapshot[applied={T|T < 7332696391173136384 or (T in {7332696391173136384})}]
I20260924 00:17:59.882355 19848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:59.882460 19848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:17:59.883224 19848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d39fbd88bcb2470aa8c416d52ec5a60b: Sending DeleteTablet for 3 replicas of tablet f2da981363084f1f9a5c088e7eca6283
I20260924 00:17:59.883550 19970 tablet_service.cc:1514] Processing DeleteTablet for tablet f2da981363084f1f9a5c088e7eca6283 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:59 UTC) from {username='slave'} at 127.0.0.1:51928
I20260924 00:17:59.883570 20232 tablet_service.cc:1514] Processing DeleteTablet for tablet f2da981363084f1f9a5c088e7eca6283 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:59 UTC) from {username='slave'} at 127.0.0.1:60016
I20260924 00:17:59.883776 20335 tablet_replica.cc:330] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35: stopping tablet replica
I20260924 00:17:59.883785 20101 tablet_service.cc:1514] Processing DeleteTablet for tablet f2da981363084f1f9a5c088e7eca6283 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:17:59 UTC) from {username='slave'} at 127.0.0.1:53698
I20260924 00:17:59.883864 20335 raft_consensus.cc:2229] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:17:59.883955 20335 raft_consensus.cc:2258] T f2da981363084f1f9a5c088e7eca6283 P 0baac095ad814ef8831ded62c1604f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:59.884407 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19908
I20260924 00:17:59.885649 20337 tablet_replica.cc:330] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: stopping tablet replica
I20260924 00:17:59.885774 20337 raft_consensus.cc:2229] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:59.885872 20337 raft_consensus.cc:2258] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:59.886228 20337 ts_tablet_manager.cc:1905] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:59.887666 20337 ts_tablet_manager.cc:1918] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:17:59.887766 20337 log.cc:1196] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/f2da981363084f1f9a5c088e7eca6283
I20260924 00:17:59.887989 20337 ts_tablet_manager.cc:1939] T f2da981363084f1f9a5c088e7eca6283 P c6dcda42e1af4f87a5b785d7682a0214: Deleting consensus metadata
I20260924 00:17:59.888473 19832 catalog_manager.cc:4841] TS c6dcda42e1af4f87a5b785d7682a0214 (127.1.44.130:33323): tablet f2da981363084f1f9a5c088e7eca6283 (table test-table [id=311cdf15e67940b097dbc1517b3e8e50]) successfully deleted
W20260924 00:17:59.889331 19835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:35951 (error 108)
I20260924 00:17:59.889786 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.889664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:35951 (local address 127.0.0.1:51944)
0924 00:17:59.889726 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.889761 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:35951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 00:17:59.889869 20336 tablet_replica.cc:330] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: stopping tablet replica
W20260924 00:17:59.889925 19835 catalog_manager.cc:4584] TS 0baac095ad814ef8831ded62c1604f35 (127.1.44.129:35951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2da981363084f1f9a5c088e7eca6283: Network error: Client connection negotiation failed: client connection to 127.1.44.129:35951: connect: Connection refused (error 111)
I20260924 00:17:59.889974 20336 raft_consensus.cc:2229] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:17:59.890065 20336 raft_consensus.cc:2258] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:17:59.890262 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20039
I20260924 00:17:59.895450 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20170
I20260924 00:17:59.895535 20336 ts_tablet_manager.cc:1905] T f2da981363084f1f9a5c088e7eca6283 P e96b70a5216a4f9eb5eb8da0a429112c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:17:59.900754 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 19817
I20260924 00:17:59.900938 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:17:59.900814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:34207 (local address 127.0.0.1:60024)
0924 00:17:59.900873 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 00:17:59.900898 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.131:34207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
2026-09-24T00:17:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (713 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:17:59 dist-test-slave-cm7b krb5kdc[20341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:17:59 dist-test-slave-cm7b krb5kdc[20341](info): set up 2 sockets
Sep 24 00:17:59 dist-test-slave-cm7b krb5kdc[20341](info): commencing operation
krb5kdc: starting...
W20260924 00:18:00.930907 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:01.043433 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34297
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:34297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:01.121443 20354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:01.121666 20354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:01.121703 20354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:01.122817 20354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:01.122880 20354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:01.122903 20354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:01.122925 20354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:01.122951 20354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:01.124408 20354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:34297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:34297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:01.124665 20354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:01.124874 20354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:01.129496 20360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.129549 20359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.129523 20362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:01.129757 20354 server_base.cc:1034] running on GCE node
I20260924 00:18:01.130007 20354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:01.130293 20354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:01.131438 20354 hybrid_clock.cc:648] HybridClock initialized: now 1790209081131428 us; error 32 us; skew 500 ppm
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:01.134055 20354 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:01.134538 20354 webserver.cc:460] Webserver started at http://127.1.44.190:39841/ using document root <none> and password file <none>
I20260924 00:18:01.134752 20354 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:01.134815 20354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:01.134960 20354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:01.136032 20354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "d12e2cba062a472caab3b81d89a88808"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "3bc6eda7b70dd9044ba780af901d8deb"
server_key_iv: "b44595a9ec339038cadc88b6a3152627"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.136394 20354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "d12e2cba062a472caab3b81d89a88808"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "3bc6eda7b70dd9044ba780af901d8deb"
server_key_iv: "b44595a9ec339038cadc88b6a3152627"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.137676 20354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:01.138408 20369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.138598 20354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:01.138703 20354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "d12e2cba062a472caab3b81d89a88808"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "3bc6eda7b70dd9044ba780af901d8deb"
server_key_iv: "b44595a9ec339038cadc88b6a3152627"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.138782 20354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:01.146687 20354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:01.148871 20354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:01.149024 20354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:01.153012 20354 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:34297
I20260924 00:18:01.153045 20421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:34297 every 8 connection(s)
I20260924 00:18:01.153410 20354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:01.154211 20422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:01.156418 20422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: Bootstrap starting.
I20260924 00:18:01.156983 20422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:01.157305 20422 log.cc:824] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:01.157445 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20354
I20260924 00:18:01.157605 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:01.157826 1202 external_mini_cluster.cc:1428] Setting key 11ecc78d9d27f32e618daa85ba37a7c1
I20260924 00:18:01.158062 20422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: No bootstrap required, opened a new log
I20260924 00:18:01.160001 20422 raft_consensus.cc:348] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12e2cba062a472caab3b81d89a88808" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34297 } }
I20260924 00:18:01.160181 20422 raft_consensus.cc:374] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:01.160229 20422 raft_consensus.cc:729] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12e2cba062a472caab3b81d89a88808, State: Initialized, Role: FOLLOWER
I20260924 00:18:01.160358 20422 consensus_queue.cc:260] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12e2cba062a472caab3b81d89a88808" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34297 } }
I20260924 00:18:01.160440 20422 raft_consensus.cc:388] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:01.160482 20422 raft_consensus.cc:482] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:01.160531 20422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:01.161051 20422 raft_consensus.cc:504] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12e2cba062a472caab3b81d89a88808" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34297 } }
I20260924 00:18:01.161175 20422 leader_election.cc:302] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d12e2cba062a472caab3b81d89a88808; no voters:
I20260924 00:18:01.161355 20422 leader_election.cc:288] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:01.161391 20427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:01.161504 20427 raft_consensus.cc:686] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [term 1 LEADER]: Becoming Leader. State: Replica: d12e2cba062a472caab3b81d89a88808, State: Running, Role: LEADER
I20260924 00:18:01.161610 20422 sys_catalog.cc:564] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:01.161583 20427 consensus_queue.cc:237] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12e2cba062a472caab3b81d89a88808" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34297 } }
I20260924 00:18:01.161914 20427 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d12e2cba062a472caab3b81d89a88808. Latest consensus state: current_term: 1 leader_uuid: "d12e2cba062a472caab3b81d89a88808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12e2cba062a472caab3b81d89a88808" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34297 } } }
I20260924 00:18:01.161991 20427 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:01.161926 20428 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d12e2cba062a472caab3b81d89a88808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12e2cba062a472caab3b81d89a88808" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 34297 } } }
I20260924 00:18:01.162150 20428 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808 [sys.catalog]: This master's current role is: LEADER
W20260924 00:18:01.163409 20442 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:01.163467 20442 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:01.163518 20432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:01.163691 20432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:01.168224 20432 catalog_manager.cc:1347] Generated new cluster ID: 4fb350b2f66d47b7b4205789abc06481
I20260924 00:18:01.168270 20432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:01.169092 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.158319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.1.44.190:34297)
0924 00:18:01.158501 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.158510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.158539 (+ 29us) server_negotiation.cc:407] Connection header received
0924 00:18:01.159311 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.159319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.159367 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.159502 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:01.160093 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.161504 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.162502 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.162882 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.162998 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.165563 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:01.165567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:01.165567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:01.165577 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:01.167507 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.167920 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.167923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.167924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.168022 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.168289 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.168290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.168290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.168497 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:01.168657 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.168823 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.168953 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260924 00:18:01.174986 20432 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:01.175681 20432 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:01.186362 20432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: Generated new TSK 0
I20260924 00:18:01.186522 20432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.44.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
Entry for principal kudu/127.1.44.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab.
I20260924 00:18:01.213532 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34297
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:01.293321 20448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:01.293536 20448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:01.293574 20448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:01.294687 20448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:01.294759 20448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:01.294803 20448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:18:01.296375 20448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.129.keytab
--principal=kudu/127.1.44.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:01.296636 20448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:01.296866 20448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:01.301909 20453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.301885 20454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.301882 20456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:01.302038 20448 server_base.cc:1034] running on GCE node
I20260924 00:18:01.302302 20448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:01.302491 20448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:01.303622 20448 hybrid_clock.cc:648] HybridClock initialized: now 1790209081303597 us; error 38 us; skew 500 ppm
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:01.306028 20448 init.cc:377] Logged in from keytab as kudu/127.1.44.129@KRBTEST.COM (short username kudu)
I20260924 00:18:01.306447 20448 webserver.cc:460] Webserver started at http://127.1.44.129:41253/ using document root <none> and password file <none>
I20260924 00:18:01.306665 20448 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:01.306711 20448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:01.306872 20448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:01.307976 20448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "de0579cf03b3409094d8ffd5d1ec932d"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "fde2b61d2a08e965964cff7509393639"
server_key_iv: "21971349bbe0b0fba02af85051074ddf"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.308311 20448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "de0579cf03b3409094d8ffd5d1ec932d"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "fde2b61d2a08e965964cff7509393639"
server_key_iv: "21971349bbe0b0fba02af85051074ddf"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.309561 20448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:18:01.310231 20463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.310410 20448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:01.310498 20448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "de0579cf03b3409094d8ffd5d1ec932d"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "fde2b61d2a08e965964cff7509393639"
server_key_iv: "21971349bbe0b0fba02af85051074ddf"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.310566 20448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:01.326761 20448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:01.328928 20448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:01.329078 20448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:01.329303 20448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:18:01.329612 20448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:18:01.329670 20448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.329703 20448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:18:01.329730 20448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.335336 20448 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:33399
I20260924 00:18:01.335433 20576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:33399 every 8 connection(s)
I20260924 00:18:01.335721 20448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:18:01.338220 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20448
I20260924 00:18:01.338346 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:18:01.338544 1202 external_mini_cluster.cc:1428] Setting key d7c89c370022c34fbc66d55f23131c13
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.129@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:01.345419 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.336386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:44029 (local address 127.1.44.190:34297)
0924 00:18:01.336446 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.336449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.336649 (+ 200us) server_negotiation.cc:407] Connection header received
0924 00:18:01.337193 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.337194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.337207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.337257 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:01.338203 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.338933 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.340027 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.340336 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.340408 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.342633 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:01.342636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:01.342636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:01.342652 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:01.344074 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.344482 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.344483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.344484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.344545 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.344788 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.344789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.344789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.344900 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:01.344970 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.345247 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.345330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 00:18:01.345603 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.336376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34297 (local address 127.1.44.129:44029)
0924 00:18:01.336511 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.336572 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.337070 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.337339 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.337349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.337904 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:01.338087 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.338092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.339020 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.339021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.339937 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.339939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.340433 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.340434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.340547 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.341037 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:01.341044 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:01.342531 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:01.344165 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:01.344172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:01.344174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:01.344393 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:01.344611 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:01.344611 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:01.344612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:01.344722 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:01.344972 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:01.344979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:01.345110 (+ 131us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.345215 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.345364 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260924 00:18:01.346087 20577 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34297
I20260924 00:18:01.346176 20577 heartbeater.cc:464] Registering TS with master...
I20260924 00:18:01.346360 20577 heartbeater.cc:511] Master 127.1.44.190:34297 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.44.130@KRBTEST.COM; defaulting to no policy
I20260924 00:18:01.346900 20386 ts_manager.cc:194] Registered new tserver with Master: de0579cf03b3409094d8ffd5d1ec932d (127.1.44.129:33399)
I20260924 00:18:01.347677 20386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.129@KRBTEST.COM'} at 127.1.44.129:44029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
Entry for principal kudu/127.1.44.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab.
I20260924 00:18:01.362484 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34297
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:01.443925 20582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:01.444113 20582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:01.444144 20582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:01.445199 20582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:01.445235 20582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:01.445261 20582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:18:01.446605 20582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.130.keytab
--principal=kudu/127.1.44.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:01.446794 20582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:01.446957 20582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:01.452013 20587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.452010 20590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.452001 20588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:01.452344 20582 server_base.cc:1034] running on GCE node
I20260924 00:18:01.452529 20582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:01.452788 20582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:01.453951 20582 hybrid_clock.cc:648] HybridClock initialized: now 1790209081453939 us; error 29 us; skew 500 ppm
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:01.456466 20582 init.cc:377] Logged in from keytab as kudu/127.1.44.130@KRBTEST.COM (short username kudu)
I20260924 00:18:01.456919 20582 webserver.cc:460] Webserver started at http://127.1.44.130:37323/ using document root <none> and password file <none>
I20260924 00:18:01.457137 20582 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:01.457201 20582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:01.457330 20582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:01.458319 20582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "d17f3d36c2168645af929a03c932ddd0"
server_key_iv: "5072025eb30c11736838f5ed4fab7568"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.458657 20582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "d17f3d36c2168645af929a03c932ddd0"
server_key_iv: "5072025eb30c11736838f5ed4fab7568"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.460271 20582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:01.461025 20597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.461272 20582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 00:18:01.461388 20582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "d17f3d36c2168645af929a03c932ddd0"
server_key_iv: "5072025eb30c11736838f5ed4fab7568"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.461476 20582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:01.469403 20582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:01.471156 20582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:01.471328 20582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:01.471532 20582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:18:01.471836 20582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:18:01.471885 20582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.471920 20582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:18:01.471951 20582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.477591 20582 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:42037
I20260924 00:18:01.477636 20710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:42037 every 8 connection(s)
I20260924 00:18:01.477917 20582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.130@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:01.486148 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.478480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:38607 (local address 127.1.44.190:34297)
0924 00:18:01.478520 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.478523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.478714 (+ 191us) server_negotiation.cc:407] Connection header received
0924 00:18:01.479217 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.479218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.479231 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.479293 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:01.480173 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.480672 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.481649 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.481840 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.481921 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.483929 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:01.483930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:01.483930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:01.483936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:01.485067 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.485444 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.485444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.485444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.485489 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.485687 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.485688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.485688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.485781 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:01.485828 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.486033 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.486088 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:18:01.486295 20713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.478445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34297 (local address 127.1.44.130:38607)
0924 00:18:01.478575 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.478633 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.479096 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.479357 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.479361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.479905 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:01.480078 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.480082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.480770 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.480771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.481546 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.481548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.481928 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.481930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.482044 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.482458 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:01.482465 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:01.483845 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:01.485157 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:01.485163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:01.485164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:01.485372 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:01.485541 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:01.485541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:01.485541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:01.485638 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:01.485830 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:01.485833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:01.485933 (+ 100us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.486009 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.486113 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260924 00:18:01.486764 20711 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34297
I20260924 00:18:01.486826 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20582
I20260924 00:18:01.486868 20711 heartbeater.cc:464] Registering TS with master...
I20260924 00:18:01.486932 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:18:01.487057 20711 heartbeater.cc:511] Master 127.1.44.190:34297 requested a full tablet report, sending...
I20260924 00:18:01.487124 1202 external_mini_cluster.cc:1428] Setting key fb55171ce83cac6f85b8b029e318f7fa
I20260924 00:18:01.487504 20386 ts_manager.cc:194] Registered new tserver with Master: aadfa6c93ce941c78bc6aa5e5a5d8941 (127.1.44.130:42037)
I20260924 00:18:01.487963 20386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.130@KRBTEST.COM'} at 127.1.44.130:38607
WARNING: no policy specified for kudu/127.1.44.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
Entry for principal kudu/127.1.44.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab.
I20260924 00:18:01.510354 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34297
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:01.588851 20716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:01.589030 20716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:01.589052 20716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:01.590099 20716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:01.590132 20716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:01.590158 20716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:18:01.591495 20716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.131.keytab
--principal=kudu/127.1.44.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:34297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:01.591663 20716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:01.591825 20716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:18:01.596573 20716 server_base.cc:1034] running on GCE node
W20260924 00:18:01.596555 20724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.596544 20722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.596555 20721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:01.596881 20716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:01.597114 20716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:01.598268 20716 hybrid_clock.cc:648] HybridClock initialized: now 1790209081598263 us; error 37 us; skew 500 ppm
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:01.600598 20716 init.cc:377] Logged in from keytab as kudu/127.1.44.131@KRBTEST.COM (short username kudu)
I20260924 00:18:01.600993 20716 webserver.cc:460] Webserver started at http://127.1.44.131:45877/ using document root <none> and password file <none>
I20260924 00:18:01.601219 20716 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:01.601282 20716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:01.601420 20716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:01.602375 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "c14c2f943b804dc9b272084632e473f8"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "06c3116592ceda2923cfd46df75a35b5"
server_key_iv: "fa4de3a4fb33b9cf1ca3aeffb064e0b3"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.602713 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "c14c2f943b804dc9b272084632e473f8"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "06c3116592ceda2923cfd46df75a35b5"
server_key_iv: "fa4de3a4fb33b9cf1ca3aeffb064e0b3"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.603999 20716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:01.604687 20731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.604846 20716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:01.604936 20716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "c14c2f943b804dc9b272084632e473f8"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "06c3116592ceda2923cfd46df75a35b5"
server_key_iv: "fa4de3a4fb33b9cf1ca3aeffb064e0b3"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.605010 20716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:01.612943 20716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:01.614765 20716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:01.614908 20716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:01.615109 20716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:18:01.615406 20716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:18:01.615456 20716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.615496 20716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:18:01.615525 20716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.621315 20716 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:40537
I20260924 00:18:01.621366 20844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:40537 every 8 connection(s)
I20260924 00:18:01.621637 20716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:18:01.624357 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20716
I20260924 00:18:01.624477 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:18:01.624648 1202 external_mini_cluster.cc:1428] Setting key 2ce93b4fb8e4f00309e5fe47dd701f9f
Sep 24 00:18:01 dist-test-slave-cm7b krb5kdc[20341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209081, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.131@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:01.630467 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.622238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:42801 (local address 127.1.44.190:34297)
0924 00:18:01.622282 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.622285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.622450 (+ 165us) server_negotiation.cc:407] Connection header received
0924 00:18:01.622939 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.622940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.622953 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.622998 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:01.623948 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.624521 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.625774 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.625984 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.626063 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.628055 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:01.628057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:01.628057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:01.628065 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:01.629182 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.629569 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.629570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.629571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.629629 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.629866 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.629867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.629867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.629978 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:01.630046 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.630301 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.630377 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:01.630641 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.622198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:34297 (local address 127.1.44.131:42801)
0924 00:18:01.622336 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.622377 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.622833 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.623072 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.623075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.623669 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:01.623846 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.623850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.624596 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.624597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.625679 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.625682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.626070 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.626071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.626196 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.626668 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:01.626682 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:01.627968 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:01.629285 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:01.629292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:01.629294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:01.629499 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:01.629692 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:01.629693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:01.629693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:01.629803 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:01.630051 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:01.630057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:01.630181 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.630280 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.630427 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260924 00:18:01.631106 20845 heartbeater.cc:347] Connected to a master server at 127.1.44.190:34297
I20260924 00:18:01.631196 20845 heartbeater.cc:464] Registering TS with master...
I20260924 00:18:01.631397 20845 heartbeater.cc:511] Master 127.1.44.190:34297 requested a full tablet report, sending...
I20260924 00:18:01.631759 20386 ts_manager.cc:194] Registered new tserver with Master: c14c2f943b804dc9b272084632e473f8 (127.1.44.131:40537)
I20260924 00:18:01.632077 20386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.44.131@KRBTEST.COM'} at 127.1.44.131:42801
I20260924 00:18:01.636148 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:18:01.642436 20426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.637315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.1.44.190:34297)
0924 00:18:01.637348 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.637349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.637454 (+ 105us) server_negotiation.cc:407] Connection header received
0924 00:18:01.637501 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.637502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.637509 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.637551 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:01.638029 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.638470 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.639140 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.639373 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.639424 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.640388 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:01.640389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:01.640389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:01.640394 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:01.641499 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.641814 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.641814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.641814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.641859 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:01.641984 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:01.641984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:01.641985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:01.642096 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:01.642168 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.642285 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.642343 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:18:01.643769 20386 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:18:01.645181 20386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:18:01.651554 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.647095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:42037 (local address 127.0.0.1:50180)
0924 00:18:01.647538 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.647565 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.647625 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.647825 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.647828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.647831 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:01.648047 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.648050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.649362 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.649363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.650349 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.650351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.651270 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.651271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.651350 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.651363 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.651436 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.651482 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":50,"threads_started":1}
I20260924 00:18:01.651589 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.647180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.1.44.130:42037)
0924 00:18:01.647363 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.647368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.647619 (+ 251us) server_negotiation.cc:407] Connection header received
0924 00:18:01.647687 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.647689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.647701 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.647866 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:01.648134 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.649249 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.650432 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.651153 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.651263 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.651324 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.651457 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.651509 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260924 00:18:01.651824 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.647373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.1.44.129:33399)
0924 00:18:01.647535 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.647538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.647774 (+ 236us) server_negotiation.cc:407] Connection header received
0924 00:18:01.647845 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.647848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.647865 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.647940 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:01.648140 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.649243 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.650500 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.651180 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.651285 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.651335 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.651590 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.651639 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260924 00:18:01.651746 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.647130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:33399 (local address 127.0.0.1:40760)
0924 00:18:01.647698 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.647714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.647772 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.648021 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.648022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.648024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:01.648076 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.648077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.649344 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.649345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.650437 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.650439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.651369 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.651370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.651457 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.651462 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.651512 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.651549 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":98,"threads_started":1}
I20260924 00:18:01.652520 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.647130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:40537 (local address 127.0.0.1:58032)
0924 00:18:01.647852 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.647861 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.647903 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.648143 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.648144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.648146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:01.648178 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.648179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.649073 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.649074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.650336 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.650338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.652279 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.652280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.652348 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.652353 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.652403 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.652446 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":42,"thread_start_us":94,"threads_started":1}
I20260924 00:18:01.652730 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.647339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58032 (local address 127.1.44.131:40537)
0924 00:18:01.647496 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.647500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.647963 (+ 463us) server_negotiation.cc:407] Connection header received
0924 00:18:01.647999 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.648001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.648016 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.648081 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:01.648252 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.648976 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.651272 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.651983 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.652058 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.652115 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.652619 (+ 504us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.652659 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260924 00:18:01.652706 20511 tablet_service.cc:1467] Processing CreateTablet for tablet b70c8ff49ee44e46b2215caab0f5486c (DEFAULT_TABLE table=test-table [id=e03ad94697d343808ade49dcf92c304f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:18:01.652714 20645 tablet_service.cc:1467] Processing CreateTablet for tablet b70c8ff49ee44e46b2215caab0f5486c (DEFAULT_TABLE table=test-table [id=e03ad94697d343808ade49dcf92c304f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:18:01.653026 20645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b70c8ff49ee44e46b2215caab0f5486c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:01.653026 20511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b70c8ff49ee44e46b2215caab0f5486c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:01.655771 20779 tablet_service.cc:1467] Processing CreateTablet for tablet b70c8ff49ee44e46b2215caab0f5486c (DEFAULT_TABLE table=test-table [id=e03ad94697d343808ade49dcf92c304f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:18:01.656057 20779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b70c8ff49ee44e46b2215caab0f5486c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:01.656419 20860 tablet_bootstrap.cc:492] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d: Bootstrap starting.
I20260924 00:18:01.656929 20860 tablet_bootstrap.cc:654] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:01.657449 20860 log.cc:824] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:01.657784 20859 tablet_bootstrap.cc:492] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Bootstrap starting.
I20260924 00:18:01.658326 20859 tablet_bootstrap.cc:654] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:01.658689 20862 tablet_bootstrap.cc:492] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Bootstrap starting.
I20260924 00:18:01.658878 20859 log.cc:824] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:01.659389 20862 tablet_bootstrap.cc:654] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:01.659487 20860 tablet_bootstrap.cc:492] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d: No bootstrap required, opened a new log
I20260924 00:18:01.659557 20860 ts_tablet_manager.cc:1397] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 00:18:01.659691 20859 tablet_bootstrap.cc:492] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: No bootstrap required, opened a new log
I20260924 00:18:01.659773 20859 ts_tablet_manager.cc:1397] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:18:01.659929 20862 log.cc:824] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:01.660635 20862 tablet_bootstrap.cc:492] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: No bootstrap required, opened a new log
I20260924 00:18:01.660704 20862 ts_tablet_manager.cc:1397] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:18:01.661358 20860 raft_consensus.cc:348] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.661506 20860 raft_consensus.cc:374] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:01.661549 20860 raft_consensus.cc:729] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0579cf03b3409094d8ffd5d1ec932d, State: Initialized, Role: FOLLOWER
I20260924 00:18:01.661721 20860 consensus_queue.cc:260] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.661765 20859 raft_consensus.cc:348] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.661928 20859 raft_consensus.cc:374] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:01.661968 20859 raft_consensus.cc:729] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aadfa6c93ce941c78bc6aa5e5a5d8941, State: Initialized, Role: FOLLOWER
I20260924 00:18:01.661981 20860 ts_tablet_manager.cc:1428] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 00:18:01.662102 20577 heartbeater.cc:503] Master 127.1.44.190:34297 was elected leader, sending a full tablet report...
I20260924 00:18:01.662099 20859 consensus_queue.cc:260] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.662359 20711 heartbeater.cc:503] Master 127.1.44.190:34297 was elected leader, sending a full tablet report...
I20260924 00:18:01.662356 20859 ts_tablet_manager.cc:1428] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:18:01.662779 20862 raft_consensus.cc:348] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.662950 20862 raft_consensus.cc:374] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:01.662992 20862 raft_consensus.cc:729] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14c2f943b804dc9b272084632e473f8, State: Initialized, Role: FOLLOWER
I20260924 00:18:01.663105 20862 consensus_queue.cc:260] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.663381 20862 ts_tablet_manager.cc:1428] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:18:01.663467 20845 heartbeater.cc:503] Master 127.1.44.190:34297 was elected leader, sending a full tablet report...
I20260924 00:18:01.700050 20865 raft_consensus.cc:482] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:18:01.700176 20865 raft_consensus.cc:504] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.700572 20865 leader_election.cc:288] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aadfa6c93ce941c78bc6aa5e5a5d8941 (127.1.44.130:42037), c14c2f943b804dc9b272084632e473f8 (127.1.44.131:40537)
I20260924 00:18:01.704778 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.700754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:59997 (local address 127.1.44.130:42037)
0924 00:18:01.700804 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.700806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.700863 (+ 57us) server_negotiation.cc:407] Connection header received
0924 00:18:01.700890 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.700891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.700903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.700961 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:01.701191 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.701937 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.703336 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.703918 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.703997 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.704006 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.704675 (+ 669us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.704716 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 00:18:01.704778 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.700691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:42037 (local address 127.1.44.129:59997)
0924 00:18:01.700771 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.700783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.700836 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.700968 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.700969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.700971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:01.701088 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.701091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.702029 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.702030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.703237 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.703240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.704494 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.704495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.704592 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.704600 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.704668 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.704711 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 00:18:01.705000 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.700953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:40537 (local address 127.1.44.129:49519)
0924 00:18:01.701154 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.701164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:01.701228 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:01.701399 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:01.701399 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:01.701401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:01.701479 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.701480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.702746 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.702747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.703983 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:01.703986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.704717 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.704718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:01.704789 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.704793 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:18:01.704866 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 00:18:01.704913 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
I20260924 00:18:01.705111 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70c8ff49ee44e46b2215caab0f5486c" candidate_uuid: "de0579cf03b3409094d8ffd5d1ec932d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" is_pre_election: true
I20260924 00:18:01.705272 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.701044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:49519 (local address 127.1.44.131:40537)
0924 00:18:01.701086 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.701088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.701207 (+ 119us) server_negotiation.cc:407] Connection header received
0924 00:18:01.701299 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.701300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.701310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.701339 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:01.701687 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.702621 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.704112 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.704639 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.704717 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.704725 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.705169 (+ 444us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.705204 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 00:18:01.705281 20665 raft_consensus.cc:2454] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de0579cf03b3409094d8ffd5d1ec932d in term 0.
I20260924 00:18:01.705521 20465 leader_election.cc:302] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aadfa6c93ce941c78bc6aa5e5a5d8941, de0579cf03b3409094d8ffd5d1ec932d; no voters:
I20260924 00:18:01.705650 20865 raft_consensus.cc:2781] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:18:01.705704 20865 raft_consensus.cc:482] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:18:01.705643 20799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70c8ff49ee44e46b2215caab0f5486c" candidate_uuid: "de0579cf03b3409094d8ffd5d1ec932d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14c2f943b804dc9b272084632e473f8" is_pre_election: true
I20260924 00:18:01.705740 20865 raft_consensus.cc:3037] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:01.705791 20799 raft_consensus.cc:2454] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de0579cf03b3409094d8ffd5d1ec932d in term 0.
I20260924 00:18:01.706465 20865 raft_consensus.cc:504] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.706593 20865 leader_election.cc:288] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [CANDIDATE]: Term 1 election: Requested vote from peers aadfa6c93ce941c78bc6aa5e5a5d8941 (127.1.44.130:42037), c14c2f943b804dc9b272084632e473f8 (127.1.44.131:40537)
I20260924 00:18:01.706725 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70c8ff49ee44e46b2215caab0f5486c" candidate_uuid: "de0579cf03b3409094d8ffd5d1ec932d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941"
I20260924 00:18:01.706729 20799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b70c8ff49ee44e46b2215caab0f5486c" candidate_uuid: "de0579cf03b3409094d8ffd5d1ec932d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14c2f943b804dc9b272084632e473f8"
I20260924 00:18:01.706807 20665 raft_consensus.cc:3037] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:01.706823 20799 raft_consensus.cc:3037] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:01.707594 20799 raft_consensus.cc:2454] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de0579cf03b3409094d8ffd5d1ec932d in term 1.
I20260924 00:18:01.707687 20665 raft_consensus.cc:2454] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de0579cf03b3409094d8ffd5d1ec932d in term 1.
I20260924 00:18:01.707785 20467 leader_election.cc:302] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c14c2f943b804dc9b272084632e473f8, de0579cf03b3409094d8ffd5d1ec932d; no voters:
I20260924 00:18:01.707875 20865 raft_consensus.cc:2781] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:01.708016 20865 raft_consensus.cc:686] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [term 1 LEADER]: Becoming Leader. State: Replica: de0579cf03b3409094d8ffd5d1ec932d, State: Running, Role: LEADER
I20260924 00:18:01.708132 20865 consensus_queue.cc:237] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } }
I20260924 00:18:01.708910 20386 catalog_manager.cc:5495] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d reported cstate change: term changed from 0 to 1, leader changed from <none> to de0579cf03b3409094d8ffd5d1ec932d (127.1.44.129). New cstate: current_term: 1 leader_uuid: "de0579cf03b3409094d8ffd5d1ec932d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de0579cf03b3409094d8ffd5d1ec932d" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 33399 } health_report { overall_health: HEALTHY } } }
I20260924 00:18:01.723069 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.721105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.1.44.129:33399)
0924 00:18:01.721152 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.721154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.721160 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:18:01.721184 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.721184 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.721198 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.721256 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:01.721381 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.721801 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.722483 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.722655 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.722707 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.722824 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:01.722907 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:01.722964 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.722994 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.723010 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 00:18:01.724607 20799 raft_consensus.cc:1261] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 1 FOLLOWER]: Refusing update from remote peer de0579cf03b3409094d8ffd5d1ec932d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:01.724608 20665 raft_consensus.cc:1261] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 1 FOLLOWER]: Refusing update from remote peer de0579cf03b3409094d8ffd5d1ec932d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:01.724920 20865 consensus_queue.cc:1035] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14c2f943b804dc9b272084632e473f8" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 40537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:01.725064 20869 consensus_queue.cc:1035] T b70c8ff49ee44e46b2215caab0f5486c P de0579cf03b3409094d8ffd5d1ec932d [LEADER]: Connected to new peer: Peer: permanent_uuid: "aadfa6c93ce941c78bc6aa5e5a5d8941" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:01.727239 20872 mvcc.cc:205] Tried to move back new op lower bound from 7332696398741594112 to 7332696398677045248. Current Snapshot: MvccSnapshot[applied={T|T < 7332696398741594112}]
I20260924 00:18:01.728009 20874 mvcc.cc:205] Tried to move back new op lower bound from 7332696398741594112 to 7332696398677045248. Current Snapshot: MvccSnapshot[applied={T|T < 7332696398741594112 or (T in {7332696398741594112})}]
I20260924 00:18:01.728122 20873 mvcc.cc:205] Tried to move back new op lower bound from 7332696398741594112 to 7332696398677045248. Current Snapshot: MvccSnapshot[applied={T|T < 7332696398741594112}]
I20260924 00:18:01.729693 20386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:18:01.729776 20386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:18:01.730458 20386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d12e2cba062a472caab3b81d89a88808: Sending DeleteTablet for 3 replicas of tablet b70c8ff49ee44e46b2215caab0f5486c
I20260924 00:18:01.730732 20645 tablet_service.cc:1514] Processing DeleteTablet for tablet b70c8ff49ee44e46b2215caab0f5486c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:18:01 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:50180
I20260924 00:18:01.730811 20779 tablet_service.cc:1514] Processing DeleteTablet for tablet b70c8ff49ee44e46b2215caab0f5486c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:18:01 UTC) from {username='kudu', principal='kudu/127.1.44.190@KRBTEST.COM'} at 127.0.0.1:58032
I20260924 00:18:01.731060 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20448
I20260924 00:18:01.731014 20883 tablet_replica.cc:330] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: stopping tablet replica
I20260924 00:18:01.731734 20883 raft_consensus.cc:2229] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:18:01.731808 20883 raft_consensus.cc:2258] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:18:01.731932 20882 tablet_replica.cc:330] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: stopping tablet replica
I20260924 00:18:01.732025 20882 raft_consensus.cc:2229] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:18:01.732116 20882 raft_consensus.cc:2258] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:18:01.732497 20882 ts_tablet_manager.cc:1905] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:18:01.733640 20883 ts_tablet_manager.cc:1905] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:18:01.733860 20882 ts_tablet_manager.cc:1918] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:18:01.733935 20882 log.cc:1196] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/b70c8ff49ee44e46b2215caab0f5486c
I20260924 00:18:01.734215 20882 ts_tablet_manager.cc:1939] T b70c8ff49ee44e46b2215caab0f5486c P aadfa6c93ce941c78bc6aa5e5a5d8941: Deleting consensus metadata
I20260924 00:18:01.734647 20371 catalog_manager.cc:4841] TS aadfa6c93ce941c78bc6aa5e5a5d8941 (127.1.44.130:42037): tablet b70c8ff49ee44e46b2215caab0f5486c (table test-table [id=e03ad94697d343808ade49dcf92c304f]) successfully deleted
I20260924 00:18:01.735178 20883 ts_tablet_manager.cc:1918] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:18:01.735332 20883 log.cc:1196] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/b70c8ff49ee44e46b2215caab0f5486c
I20260924 00:18:01.735576 20883 ts_tablet_manager.cc:1939] T b70c8ff49ee44e46b2215caab0f5486c P c14c2f943b804dc9b272084632e473f8: Deleting consensus metadata
I20260924 00:18:01.735948 20373 catalog_manager.cc:4841] TS c14c2f943b804dc9b272084632e473f8 (127.1.44.131:40537): tablet b70c8ff49ee44e46b2215caab0f5486c (table test-table [id=e03ad94697d343808ade49dcf92c304f]) successfully deleted
W20260924 00:18:01.737476 20372 connection.cc:535] client connection to 127.1.44.129:33399 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 00:18:01.737555 20372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 00:18:01.737995 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20582
I20260924 00:18:01.737996 20853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.737866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:33399 (local address 127.0.0.1:40784)
0924 00:18:01.737926 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:01.737967 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:33399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 00:18:01.740139 20372 catalog_manager.cc:4584] TS de0579cf03b3409094d8ffd5d1ec932d (127.1.44.129:33399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b70c8ff49ee44e46b2215caab0f5486c: Network error: Client connection negotiation failed: client connection to 127.1.44.129:33399: connect: Connection refused (error 111)
I20260924 00:18:01.742543 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20716
I20260924 00:18:01.748023 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20354
2026-09-24T00:18:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1848 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T00:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:01Z Disabled control of system clock
I20260924 00:18:01.775300 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:41105
--webserver_interface=127.1.44.190
--webserver_port=0
--builtin_ntp_servers=127.1.44.148:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:41105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:18:01.853039 20888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:01.853255 20888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:01.853273 20888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:01.854315 20888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:01.854347 20888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:01.854360 20888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:01.854372 20888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:01.854384 20888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:01.855633 20888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:41105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:41105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:01.855803 20888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:01.856009 20888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:01.861831 20896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.862066 20893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:01.861825 20894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:01.862510 20888 server_base.cc:1034] running on GCE node
I20260924 00:18:01.862741 20888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:01.863010 20888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:01.864166 20888 hybrid_clock.cc:648] HybridClock initialized: now 1790209081864153 us; error 38 us; skew 500 ppm
I20260924 00:18:01.865360 20888 webserver.cc:460] Webserver started at http://127.1.44.190:37643/ using document root <none> and password file <none>
I20260924 00:18:01.865589 20888 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:01.865654 20888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:01.865787 20888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:01.866798 20888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "0f5bb36bae2149db962aa3017d6117a1"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "ded10d6d6291b7d2bfdb7fb96b450f3d"
server_key_iv: "0e65ee916712ecfb9504b7b4d0025d7c"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.867141 20888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "0f5bb36bae2149db962aa3017d6117a1"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "ded10d6d6291b7d2bfdb7fb96b450f3d"
server_key_iv: "0e65ee916712ecfb9504b7b4d0025d7c"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.868490 20888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:01.869182 20902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:01.869359 20888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:01.869452 20888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "0f5bb36bae2149db962aa3017d6117a1"
format_stamp: "Formatted at 2026-09-24 00:18:01 on dist-test-slave-cm7b"
server_key: "ded10d6d6291b7d2bfdb7fb96b450f3d"
server_key_iv: "0e65ee916712ecfb9504b7b4d0025d7c"
server_key_version: "encryptionkey@0"
I20260924 00:18:01.869525 20888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:01.887969 20888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:01.888258 20888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:01.888401 20888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:01.892045 20888 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:41105
I20260924 00:18:01.892107 20954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:41105 every 8 connection(s)
I20260924 00:18:01.892436 20888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:01.893371 20955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:01.895560 20955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1: Bootstrap starting.
I20260924 00:18:01.896113 20955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:01.896394 20955 log.cc:824] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:01.897073 20955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1: No bootstrap required, opened a new log
I20260924 00:18:01.898629 20955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5bb36bae2149db962aa3017d6117a1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41105 } }
I20260924 00:18:01.898770 20955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:01.898816 20955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5bb36bae2149db962aa3017d6117a1, State: Initialized, Role: FOLLOWER
I20260924 00:18:01.898954 20955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5bb36bae2149db962aa3017d6117a1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41105 } }
I20260924 00:18:01.899050 20955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:01.899086 20955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:01.899134 20955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:01.899137 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20888
I20260924 00:18:01.899292 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:01.899471 1202 external_mini_cluster.cc:1428] Setting key f4fb274748bb9df895f15593416f2517
I20260924 00:18:01.899827 20955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5bb36bae2149db962aa3017d6117a1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41105 } }
I20260924 00:18:01.899946 20955 leader_election.cc:302] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f5bb36bae2149db962aa3017d6117a1; no voters:
I20260924 00:18:01.900105 20955 leader_election.cc:288] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:01.900230 20960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:01.900379 20955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:01.900403 20960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [term 1 LEADER]: Becoming Leader. State: Replica: 0f5bb36bae2149db962aa3017d6117a1, State: Running, Role: LEADER
I20260924 00:18:01.900477 20960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5bb36bae2149db962aa3017d6117a1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41105 } }
I20260924 00:18:01.900791 20960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f5bb36bae2149db962aa3017d6117a1. Latest consensus state: current_term: 1 leader_uuid: "0f5bb36bae2149db962aa3017d6117a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5bb36bae2149db962aa3017d6117a1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41105 } } }
I20260924 00:18:01.900877 20960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:01.900993 20961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f5bb36bae2149db962aa3017d6117a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f5bb36bae2149db962aa3017d6117a1" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 41105 } } }
I20260924 00:18:01.901082 20967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:01.901594 20967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:01.902236 20961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:01.903154 20967 catalog_manager.cc:1347] Generated new cluster ID: 2d9c6e8a1e2c4de8b0f64ae556c6171b
I20260924 00:18:01.903223 20967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:01.905269 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:01.899942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.1.44.190:41105)
0924 00:18:01.900157 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:01.900166 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:01.900204 (+ 38us) server_negotiation.cc:407] Connection header received
0924 00:18:01.900321 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:01.900328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:01.900393 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:01.900773 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:01.900877 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.902341 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.903365 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:01.903949 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:01.904059 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:01.904632 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:01.904636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:01.904636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:01.904834 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:01.904874 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:01.904887 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:01.904950 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:01.905009 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 00:18:01.905078 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260924 00:18:01.906962 20967 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:01.907503 20967 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:01.913781 20967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1: Generated new TSK 0
I20260924 00:18:01.913936 20967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:01.917841 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.129:0
--local_ip_for_outbound_sockets=127.1.44.129
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41105
--builtin_ntp_servers=127.1.44.148:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 00:18:02.013629 20979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:02.013839 20979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:02.013859 20979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:02.014987 20979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:02.015030 20979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:02.015056 20979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.129
I20260924 00:18:02.016654 20979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.44.129
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.129
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:02.016886 20979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:02.017180 20979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:02.022971 20987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:02.022966 20985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:02.023224 20979 server_base.cc:1034] running on GCE node
W20260924 00:18:02.022964 20984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:02.023531 20979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:02.023794 20979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:02.024945 20979 hybrid_clock.cc:648] HybridClock initialized: now 1790209082024925 us; error 35 us; skew 500 ppm
I20260924 00:18:02.026173 20979 webserver.cc:460] Webserver started at http://127.1.44.129:34619/ using document root <none> and password file <none>
I20260924 00:18:02.026418 20979 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:02.026484 20979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:02.026614 20979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:02.027714 20979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data/instance:
uuid: "3ac3810becaa4d62810c6888b28175cb"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "7c3f7fcf57d438369c94e9f2b7336662"
server_key_iv: "0b433594ea72a2e5cf0cd6d91b3f30aa"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.028067 20979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance:
uuid: "3ac3810becaa4d62810c6888b28175cb"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "7c3f7fcf57d438369c94e9f2b7336662"
server_key_iv: "0b433594ea72a2e5cf0cd6d91b3f30aa"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.029333 20979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 00:18:02.030057 20993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.030275 20979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:02.030378 20979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal
uuid: "3ac3810becaa4d62810c6888b28175cb"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "7c3f7fcf57d438369c94e9f2b7336662"
server_key_iv: "0b433594ea72a2e5cf0cd6d91b3f30aa"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.030452 20979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:02.054545 20979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:02.054876 20979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:02.055012 20979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:02.055302 20979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:18:02.055681 20979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:18:02.055728 20979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.055759 20979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:18:02.055792 20979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.061503 20979 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.129:41809
I20260924 00:18:02.061609 21106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.129:41809 every 8 connection(s)
I20260924 00:18:02.061908 20979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/data/info.pb
I20260924 00:18:02.063184 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 20979
I20260924 00:18:02.063369 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-0/wal/instance
I20260924 00:18:02.063553 1202 external_mini_cluster.cc:1428] Setting key 561555e57dfe121cb6bec3d89d194c48
I20260924 00:18:02.064653 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.130:0
--local_ip_for_outbound_sockets=127.1.44.130
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41105
--builtin_ntp_servers=127.1.44.148:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 00:18:02.067929 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.062734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.129:37693 (local address 127.1.44.190:41105)
0924 00:18:02.062817 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.062819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.063038 (+ 219us) server_negotiation.cc:407] Connection header received
0924 00:18:02.063625 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.063627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.063642 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.063722 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:02.064589 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.065344 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.066618 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.066812 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.066870 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.067711 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:02.067712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:02.067712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:02.067718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:02.067733 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:02.067737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:02.067782 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.067846 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.067872 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 00:18:02.068217 21109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.062683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:41105 (local address 127.1.44.129:37693)
0924 00:18:02.062880 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.062920 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.063523 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.063811 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.063815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.063826 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:02.064480 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.064485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.065432 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.065432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.066526 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.066528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.066898 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.066899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.067020 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.067557 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:18:02.067558 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:02.067595 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:18:02.067624 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:02.067785 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:02.067788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:02.067791 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.067852 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.067957 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260924 00:18:02.068864 21107 heartbeater.cc:347] Connected to a master server at 127.1.44.190:41105
I20260924 00:18:02.068966 21107 heartbeater.cc:464] Registering TS with master...
I20260924 00:18:02.069183 21107 heartbeater.cc:511] Master 127.1.44.190:41105 requested a full tablet report, sending...
I20260924 00:18:02.069720 20919 ts_manager.cc:194] Registered new tserver with Master: 3ac3810becaa4d62810c6888b28175cb (127.1.44.129:41809)
I20260924 00:18:02.070680 20919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.129:37693
W20260924 00:18:02.144351 21110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:02.144541 21110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:02.144562 21110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:02.145608 21110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:02.145641 21110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:02.145665 21110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.130
I20260924 00:18:02.146914 21110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.44.130
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.130
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:02.147091 21110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:02.147395 21110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:02.152079 21116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:02.152069 21115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:02.152069 21118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:02.152264 21110 server_base.cc:1034] running on GCE node
I20260924 00:18:02.152570 21110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:02.152809 21110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:02.153956 21110 hybrid_clock.cc:648] HybridClock initialized: now 1790209082153937 us; error 27 us; skew 500 ppm
I20260924 00:18:02.155084 21110 webserver.cc:460] Webserver started at http://127.1.44.130:36911/ using document root <none> and password file <none>
I20260924 00:18:02.155347 21110 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:02.155416 21110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:02.155544 21110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:02.156576 21110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data/instance:
uuid: "f2cf4cb459c0415299e4dec598a7bb79"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "c5ddac7de27fbcaee8782ba856c725e4"
server_key_iv: "fee6cb5123180b19789581d1c7d66f50"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.156920 21110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance:
uuid: "f2cf4cb459c0415299e4dec598a7bb79"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "c5ddac7de27fbcaee8782ba856c725e4"
server_key_iv: "fee6cb5123180b19789581d1c7d66f50"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.158172 21110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:18:02.158833 21124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.159000 21110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:02.159090 21110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal
uuid: "f2cf4cb459c0415299e4dec598a7bb79"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "c5ddac7de27fbcaee8782ba856c725e4"
server_key_iv: "fee6cb5123180b19789581d1c7d66f50"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.159173 21110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:02.171406 21110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:02.171666 21110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:02.171805 21110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:02.172012 21110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:18:02.172361 21110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:18:02.172410 21110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.172451 21110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:18:02.172477 21110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.177829 21110 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.130:34223
I20260924 00:18:02.177882 21237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.130:34223 every 8 connection(s)
I20260924 00:18:02.178195 21110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/data/info.pb
I20260924 00:18:02.180042 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21110
I20260924 00:18:02.180160 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal/instance
I20260924 00:18:02.180375 1202 external_mini_cluster.cc:1428] Setting key eff78657c8559684c25201827ced0fce
I20260924 00:18:02.181293 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.44.131:0
--local_ip_for_outbound_sockets=127.1.44.131
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41105
--builtin_ntp_servers=127.1.44.148:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 00:18:02.183686 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.178889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:36453 (local address 127.1.44.190:41105)
0924 00:18:02.178951 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.178953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.179234 (+ 281us) server_negotiation.cc:407] Connection header received
0924 00:18:02.179717 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.179718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.179728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.179793 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:02.180689 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.181175 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.182230 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.182538 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.182604 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.183406 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:02.183408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:02.183408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:02.183413 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:02.183429 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:02.183434 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:02.183494 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.183543 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.183573 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:18:02.183804 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.178883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:41105 (local address 127.1.44.130:36453)
0924 00:18:02.179043 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.179080 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.179619 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.179861 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.179865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.179877 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:02.180583 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.180590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.181263 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.181265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.182135 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.182137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.182610 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.182611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.182725 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.183292 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:18:02.183293 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:02.183319 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:18:02.183341 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:02.183495 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:02.183497 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:02.183499 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.183544 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.183616 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260924 00:18:02.184279 21238 heartbeater.cc:347] Connected to a master server at 127.1.44.190:41105
I20260924 00:18:02.184384 21238 heartbeater.cc:464] Registering TS with master...
I20260924 00:18:02.184553 21238 heartbeater.cc:511] Master 127.1.44.190:41105 requested a full tablet report, sending...
I20260924 00:18:02.184938 20919 ts_manager.cc:194] Registered new tserver with Master: f2cf4cb459c0415299e4dec598a7bb79 (127.1.44.130:34223)
I20260924 00:18:02.185303 20919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.130:36453
W20260924 00:18:02.260756 21241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:02.260941 21241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:02.260960 21241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:02.262141 21241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:02.262178 21241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:02.262202 21241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.44.131
I20260924 00:18:02.263597 21241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.44.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.44.131
--webserver_port=0
--tserver_master_addrs=127.1.44.190:41105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.44.131
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:02.263815 21241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:02.264087 21241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:02.268811 21249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:02.268822 21247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:02.268821 21246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:02.269039 21241 server_base.cc:1034] running on GCE node
I20260924 00:18:02.269304 21241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:02.269526 21241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:02.270670 21241 hybrid_clock.cc:648] HybridClock initialized: now 1790209082270654 us; error 28 us; skew 500 ppm
I20260924 00:18:02.271834 21241 webserver.cc:460] Webserver started at http://127.1.44.131:44843/ using document root <none> and password file <none>
I20260924 00:18:02.272053 21241 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:02.272117 21241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:02.272249 21241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:02.273255 21241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data/instance:
uuid: "f458ce1fd6ca40be90a3c89cc91da82a"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "760782010904c6f46e54cba161c8ecf4"
server_key_iv: "d0fa59a8fc2cd9f4ddb03a0b372c0fb6"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.273593 21241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance:
uuid: "f458ce1fd6ca40be90a3c89cc91da82a"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "760782010904c6f46e54cba161c8ecf4"
server_key_iv: "d0fa59a8fc2cd9f4ddb03a0b372c0fb6"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.274801 21241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:02.275545 21255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.275724 21241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.275821 21241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal
uuid: "f458ce1fd6ca40be90a3c89cc91da82a"
format_stamp: "Formatted at 2026-09-24 00:18:02 on dist-test-slave-cm7b"
server_key: "760782010904c6f46e54cba161c8ecf4"
server_key_iv: "d0fa59a8fc2cd9f4ddb03a0b372c0fb6"
server_key_version: "encryptionkey@0"
I20260924 00:18:02.275897 21241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:02.292932 21241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:02.293202 21241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:02.293340 21241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:02.293540 21241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 00:18:02.293840 21241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 00:18:02.293889 21241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.293929 21241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 00:18:02.293956 21241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:02.299767 21241 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.131:45829
I20260924 00:18:02.299853 21368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.131:45829 every 8 connection(s)
I20260924 00:18:02.300145 21241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/data/info.pb
I20260924 00:18:02.305213 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.300803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.131:55239 (local address 127.1.44.190:41105)
0924 00:18:02.300861 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.300863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.301160 (+ 297us) server_negotiation.cc:407] Connection header received
0924 00:18:02.301624 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.301625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.301635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.301681 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:02.302459 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.302962 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.303977 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.304189 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.304249 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.304970 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:02.304971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:02.304971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:02.304975 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:02.304990 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:02.304994 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:02.305049 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.305124 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.305151 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 00:18:02.305380 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.300837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:41105 (local address 127.1.44.131:55239)
0924 00:18:02.301020 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.301066 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.301541 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.301741 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.301744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.301752 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:02.302360 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.302365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.303073 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.303074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.303886 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.303888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.304262 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.304263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.304368 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.304827 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 00:18:02.304827 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:02.304853 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 00:18:02.304874 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:02.305055 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:02.305059 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:02.305061 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.305112 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.305191 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260924 00:18:02.305790 21369 heartbeater.cc:347] Connected to a master server at 127.1.44.190:41105
I20260924 00:18:02.305864 21369 heartbeater.cc:464] Registering TS with master...
I20260924 00:18:02.306048 21369 heartbeater.cc:511] Master 127.1.44.190:41105 requested a full tablet report, sending...
I20260924 00:18:02.306191 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21241
I20260924 00:18:02.306313 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal/instance
I20260924 00:18:02.306389 20919 ts_manager.cc:194] Registered new tserver with Master: f458ce1fd6ca40be90a3c89cc91da82a (127.1.44.131:45829)
I20260924 00:18:02.306519 1202 external_mini_cluster.cc:1428] Setting key 5c2da82b232eecde447ee18b4be2c6de
I20260924 00:18:02.306852 20919 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.44.131:55239
I20260924 00:18:02.307382 1202 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 00:18:02.311115 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.308683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40522 (local address 127.1.44.190:41105)
0924 00:18:02.308742 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.308744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.308852 (+ 108us) server_negotiation.cc:407] Connection header received
0924 00:18:02.308890 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.308890 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.308900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.308944 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:02.309049 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.309490 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.310175 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.310353 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.310400 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.310914 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:02.310915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:02.310915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:02.310919 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:02.310930 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:02.310934 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:02.310974 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.311009 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.311028 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 00:18:02.312446 20919 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 00:18:02.313905 20919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 00:18:02.319870 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.316008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.130:34223 (local address 127.0.0.1:46960)
0924 00:18:02.316155 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.316164 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.316214 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.316554 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.316555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.316556 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:02.316605 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.316606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.317532 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.317533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.318782 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.318784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.319526 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.319527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.319604 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.319617 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.319710 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.319774 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":45,"thread_start_us":91,"threads_started":1}
I20260924 00:18:02.319977 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.315793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:45829 (local address 127.0.0.1:45154)
0924 00:18:02.316142 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.316152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.316214 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.316462 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.316463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.316465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:02.316500 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.316501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.317759 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.317760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.318871 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.318873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.319728 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.319729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.319786 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.319789 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.319837 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.319876 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":37,"thread_start_us":96,"threads_started":1}
I20260924 00:18:02.319988 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.316029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.1.44.131:45829)
0924 00:18:02.316314 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.316319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.316328 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:18:02.316362 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.316364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.316378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.316457 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:02.316580 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.317667 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.318941 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.319634 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.319724 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.319797 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.319864 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.319904 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":65,"threads_started":1}
I20260924 00:18:02.320150 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.315791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41809 (local address 127.0.0.1:52174)
0924 00:18:02.315951 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.315987 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.316062 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.316372 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.316375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.316377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:02.316450 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.316453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.317778 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.317779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.318522 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.318524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.319579 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.319580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.319645 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.319649 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.319697 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.320074 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":4,"thread_start_us":59,"threads_started":1}
I20260924 00:18:02.320377 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.315866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.1.44.129:41809)
0924 00:18:02.316002 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.316007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.316040 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:18:02.316193 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.316207 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.316223 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.316297 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:02.316536 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.317691 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.318639 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.319394 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.319461 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.319544 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.320265 (+ 721us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.320304 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260924 00:18:02.319893 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.316227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.1.44.130:34223)
0924 00:18:02.316374 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.316378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.316387 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:18:02.316422 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.316424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.316438 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.316534 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:02.316661 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.317450 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.318876 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.319433 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.319523 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.319608 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.319757 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.319795 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260924 00:18:02.320676 21303 tablet_service.cc:1467] Processing CreateTablet for tablet 2b93ade36c9e4fc1a45c98de00873a2b (DEFAULT_TABLE table=test-table [id=23f836951e6a48d6a3fc8dc76f1c6420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:18:02.321035 21303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b93ade36c9e4fc1a45c98de00873a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:02.321101 21041 tablet_service.cc:1467] Processing CreateTablet for tablet 2b93ade36c9e4fc1a45c98de00873a2b (DEFAULT_TABLE table=test-table [id=23f836951e6a48d6a3fc8dc76f1c6420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:18:02.321112 21172 tablet_service.cc:1467] Processing CreateTablet for tablet 2b93ade36c9e4fc1a45c98de00873a2b (DEFAULT_TABLE table=test-table [id=23f836951e6a48d6a3fc8dc76f1c6420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 00:18:02.321399 21041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b93ade36c9e4fc1a45c98de00873a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:02.321453 21172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b93ade36c9e4fc1a45c98de00873a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:02.324101 21383 tablet_bootstrap.cc:492] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Bootstrap starting.
I20260924 00:18:02.324473 21385 tablet_bootstrap.cc:492] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: Bootstrap starting.
I20260924 00:18:02.324529 21384 tablet_bootstrap.cc:492] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Bootstrap starting.
I20260924 00:18:02.324769 21383 tablet_bootstrap.cc:654] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:02.325050 21383 log.cc:824] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:02.325115 21385 tablet_bootstrap.cc:654] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:02.325115 21384 tablet_bootstrap.cc:654] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:02.325436 21385 log.cc:824] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:02.325436 21384 log.cc:824] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:02.325762 21383 tablet_bootstrap.cc:492] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: No bootstrap required, opened a new log
I20260924 00:18:02.325837 21383 ts_tablet_manager.cc:1397] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:18:02.326148 21384 tablet_bootstrap.cc:492] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: No bootstrap required, opened a new log
I20260924 00:18:02.326242 21384 ts_tablet_manager.cc:1397] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 00:18:02.326467 21385 tablet_bootstrap.cc:492] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: No bootstrap required, opened a new log
I20260924 00:18:02.326567 21385 ts_tablet_manager.cc:1397] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 00:18:02.327826 21383 raft_consensus.cc:348] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.327950 21383 raft_consensus.cc:374] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:02.327972 21383 raft_consensus.cc:729] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2cf4cb459c0415299e4dec598a7bb79, State: Initialized, Role: FOLLOWER
I20260924 00:18:02.328084 21383 consensus_queue.cc:260] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.328344 21383 ts_tablet_manager.cc:1428] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 00:18:02.328358 21238 heartbeater.cc:503] Master 127.1.44.190:41105 was elected leader, sending a full tablet report...
I20260924 00:18:02.328804 21384 raft_consensus.cc:348] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.328955 21384 raft_consensus.cc:374] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:02.328997 21384 raft_consensus.cc:729] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f458ce1fd6ca40be90a3c89cc91da82a, State: Initialized, Role: FOLLOWER
I20260924 00:18:02.329129 21384 consensus_queue.cc:260] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.329382 21384 ts_tablet_manager.cc:1428] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 00:18:02.329489 21369 heartbeater.cc:503] Master 127.1.44.190:41105 was elected leader, sending a full tablet report...
I20260924 00:18:02.329702 21385 raft_consensus.cc:348] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.329845 21385 raft_consensus.cc:374] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:02.329903 21385 raft_consensus.cc:729] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac3810becaa4d62810c6888b28175cb, State: Initialized, Role: FOLLOWER
I20260924 00:18:02.330015 21385 consensus_queue.cc:260] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.330211 21385 ts_tablet_manager.cc:1428] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260924 00:18:02.330288 21107 heartbeater.cc:503] Master 127.1.44.190:41105 was elected leader, sending a full tablet report...
W20260924 00:18:02.428755 21239 tablet.cc:2369] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 00:18:02.550709 21370 tablet.cc:2369] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:18:02.550889 21389 raft_consensus.cc:482] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 00:18:02.551028 21389 raft_consensus.cc:504] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.551424 21389 leader_election.cc:288] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ac3810becaa4d62810c6888b28175cb (127.1.44.129:41809), f458ce1fd6ca40be90a3c89cc91da82a (127.1.44.131:45829)
I20260924 00:18:02.554082 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.551582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41809 (local address 127.1.44.130:32875)
0924 00:18:02.551624 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.551637 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.551683 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.551863 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.551865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.551866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:02.551938 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.551940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.552566 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.552567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.553308 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.553310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.553834 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.553835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.553909 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.553912 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.553968 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.554012 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 00:18:02.554083 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.551622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:32875 (local address 127.1.44.129:41809)
0924 00:18:02.551667 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.551669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.551681 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:18:02.551778 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.551778 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.551786 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.551856 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:02.552008 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.552487 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.553404 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.553752 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.553834 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.553842 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.553987 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.554025 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:18:02.554234 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.551695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.44.130:58945 (local address 127.1.44.131:45829)
0924 00:18:02.551739 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.551740 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.551894 (+ 154us) server_negotiation.cc:407] Connection header received
0924 00:18:02.551955 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.551956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.551965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.552032 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 00:18:02.552177 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.552633 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.553562 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.553910 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.553986 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.553993 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.554131 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.554168 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 00:18:02.554250 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.551590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.131:45829 (local address 127.1.44.130:58945)
0924 00:18:02.551840 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.551848 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:02.551898 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:02.552035 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:02.552036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:02.552037 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 00:18:02.552104 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.552106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.552731 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.552732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.553466 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:02.553467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.553985 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.553986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:02.554055 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.554058 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 00:18:02.554126 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 00:18:02.554162 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":67,"thread_start_us":135,"threads_started":1}
I20260924 00:18:02.554419 21061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b93ade36c9e4fc1a45c98de00873a2b" candidate_uuid: "f2cf4cb459c0415299e4dec598a7bb79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac3810becaa4d62810c6888b28175cb" is_pre_election: true
I20260924 00:18:02.554534 21323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b93ade36c9e4fc1a45c98de00873a2b" candidate_uuid: "f2cf4cb459c0415299e4dec598a7bb79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" is_pre_election: true
I20260924 00:18:02.554586 21061 raft_consensus.cc:2454] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2cf4cb459c0415299e4dec598a7bb79 in term 0.
I20260924 00:18:02.554666 21323 raft_consensus.cc:2454] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2cf4cb459c0415299e4dec598a7bb79 in term 0.
I20260924 00:18:02.554826 21127 leader_election.cc:302] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ac3810becaa4d62810c6888b28175cb, f2cf4cb459c0415299e4dec598a7bb79; no voters:
I20260924 00:18:02.554952 21389 raft_consensus.cc:2781] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 00:18:02.555011 21389 raft_consensus.cc:482] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 00:18:02.555027 21389 raft_consensus.cc:3037] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:02.555757 21389 raft_consensus.cc:504] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.555886 21389 leader_election.cc:288] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [CANDIDATE]: Term 1 election: Requested vote from peers 3ac3810becaa4d62810c6888b28175cb (127.1.44.129:41809), f458ce1fd6ca40be90a3c89cc91da82a (127.1.44.131:45829)
I20260924 00:18:02.556017 21323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b93ade36c9e4fc1a45c98de00873a2b" candidate_uuid: "f2cf4cb459c0415299e4dec598a7bb79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f458ce1fd6ca40be90a3c89cc91da82a"
I20260924 00:18:02.556039 21061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b93ade36c9e4fc1a45c98de00873a2b" candidate_uuid: "f2cf4cb459c0415299e4dec598a7bb79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ac3810becaa4d62810c6888b28175cb"
I20260924 00:18:02.556104 21061 raft_consensus.cc:3037] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:02.556104 21323 raft_consensus.cc:3037] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:02.557004 21061 raft_consensus.cc:2454] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2cf4cb459c0415299e4dec598a7bb79 in term 1.
I20260924 00:18:02.557004 21323 raft_consensus.cc:2454] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2cf4cb459c0415299e4dec598a7bb79 in term 1.
I20260924 00:18:02.557183 21126 leader_election.cc:302] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f2cf4cb459c0415299e4dec598a7bb79, f458ce1fd6ca40be90a3c89cc91da82a; no voters:
I20260924 00:18:02.557299 21389 raft_consensus.cc:2781] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:02.557462 21389 raft_consensus.cc:686] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 1 LEADER]: Becoming Leader. State: Replica: f2cf4cb459c0415299e4dec598a7bb79, State: Running, Role: LEADER
I20260924 00:18:02.557559 21389 consensus_queue.cc:237] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } }
I20260924 00:18:02.558296 20919 catalog_manager.cc:5495] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2cf4cb459c0415299e4dec598a7bb79 (127.1.44.130). New cstate: current_term: 1 leader_uuid: "f2cf4cb459c0415299e4dec598a7bb79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2cf4cb459c0415299e4dec598a7bb79" member_type: VOTER last_known_addr { host: "127.1.44.130" port: 34223 } health_report { overall_health: HEALTHY } } }
W20260924 00:18:02.562534 21108 tablet.cc:2369] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 00:18:02.600976 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.598468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.1.44.130:34223)
0924 00:18:02.598529 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:02.598532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:02.598545 (+ 13us) server_negotiation.cc:407] Connection header received
0924 00:18:02.598593 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:02.598594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:02.598616 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:02.598686 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:02.598853 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.599381 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.600165 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:02.600354 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:02.600435 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:02.600640 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:02.600740 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:02.600804 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:02.600836 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 00:18:02.600851 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 00:18:02.602831 21323 raft_consensus.cc:1261] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 1 FOLLOWER]: Refusing update from remote peer f2cf4cb459c0415299e4dec598a7bb79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:02.602831 21061 raft_consensus.cc:1261] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 1 FOLLOWER]: Refusing update from remote peer f2cf4cb459c0415299e4dec598a7bb79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:02.603217 21389 consensus_queue.cc:1035] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ac3810becaa4d62810c6888b28175cb" member_type: VOTER last_known_addr { host: "127.1.44.129" port: 41809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:02.603356 21393 consensus_queue.cc:1035] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f458ce1fd6ca40be90a3c89cc91da82a" member_type: VOTER last_known_addr { host: "127.1.44.131" port: 45829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:02.605641 21396 mvcc.cc:205] Tried to move back new op lower bound from 7332696402338377728 to 7332696402156187648. Current Snapshot: MvccSnapshot[applied={T|T < 7332696402338377728}]
I20260924 00:18:02.605792 21397 mvcc.cc:205] Tried to move back new op lower bound from 7332696402338377728 to 7332696402156187648. Current Snapshot: MvccSnapshot[applied={T|T < 7332696402338377728}]
I20260924 00:18:02.605892 21398 mvcc.cc:205] Tried to move back new op lower bound from 7332696402338377728 to 7332696402156187648. Current Snapshot: MvccSnapshot[applied={T|T < 7332696402338377728}]
I20260924 00:18:02.608186 20918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:18:02.608294 20918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 00:18:02.608994 20918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f5bb36bae2149db962aa3017d6117a1: Sending DeleteTablet for 3 replicas of tablet 2b93ade36c9e4fc1a45c98de00873a2b
I20260924 00:18:02.609328 21172 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b93ade36c9e4fc1a45c98de00873a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:18:02 UTC) from {username='slave'} at 127.0.0.1:46960
I20260924 00:18:02.609436 21041 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b93ade36c9e4fc1a45c98de00873a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:18:02 UTC) from {username='slave'} at 127.0.0.1:52174
I20260924 00:18:02.609633 21406 tablet_replica.cc:330] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: stopping tablet replica
I20260924 00:18:02.609651 21407 tablet_replica.cc:330] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb: stopping tablet replica
I20260924 00:18:02.609745 21407 raft_consensus.cc:2229] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:18:02.609745 21303 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b93ade36c9e4fc1a45c98de00873a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 00:18:02 UTC) from {username='slave'} at 127.0.0.1:45154
I20260924 00:18:02.609869 21407 raft_consensus.cc:2258] T 2b93ade36c9e4fc1a45c98de00873a2b P 3ac3810becaa4d62810c6888b28175cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:18:02.609934 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20979
I20260924 00:18:02.609774 21406 raft_consensus.cc:2229] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 1 LEADER]: Raft consensus shutting down.
I20260924 00:18:02.610133 21406 raft_consensus.cc:2258] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:18:02.610680 21406 ts_tablet_manager.cc:1905] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:18:02.612536 21406 ts_tablet_manager.cc:1918] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:18:02.612609 21406 log.cc:1196] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-1/wal/wals/2b93ade36c9e4fc1a45c98de00873a2b
I20260924 00:18:02.612866 21406 ts_tablet_manager.cc:1939] T 2b93ade36c9e4fc1a45c98de00873a2b P f2cf4cb459c0415299e4dec598a7bb79: Deleting consensus metadata
I20260924 00:18:02.613269 20905 catalog_manager.cc:4841] TS f2cf4cb459c0415299e4dec598a7bb79 (127.1.44.130:34223): tablet 2b93ade36c9e4fc1a45c98de00873a2b (table test-table [id=23f836951e6a48d6a3fc8dc76f1c6420]) successfully deleted
W20260924 00:18:02.615175 20905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.44.129:41809 (error 108)
I20260924 00:18:02.615648 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21110
I20260924 00:18:02.615670 21408 tablet_replica.cc:330] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: stopping tablet replica
I20260924 00:18:02.615763 21408 raft_consensus.cc:2229] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 00:18:02.615861 21408 raft_consensus.cc:2258] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 00:18:02.616438 21408 ts_tablet_manager.cc:1905] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 00:18:02.618311 21408 ts_tablet_manager.cc:1918] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 00:18:02.618384 21408 log.cc:1196] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Deleting WAL directory at /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790208846277299-1202-0/minicluster-data/ts-2/wal/wals/2b93ade36c9e4fc1a45c98de00873a2b
I20260924 00:18:02.618618 21408 ts_tablet_manager.cc:1939] T 2b93ade36c9e4fc1a45c98de00873a2b P f458ce1fd6ca40be90a3c89cc91da82a: Deleting consensus metadata
I20260924 00:18:02.619004 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:02.618834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.129:41809 (local address 127.0.0.1:44620)
0924 00:18:02.618898 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:02.618945 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 00:18:02.619343 20904 catalog_manager.cc:4841] TS f458ce1fd6ca40be90a3c89cc91da82a (127.1.44.131:45829): tablet 2b93ade36c9e4fc1a45c98de00873a2b (table test-table [id=23f836951e6a48d6a3fc8dc76f1c6420]) successfully deleted
W20260924 00:18:02.621924 20905 catalog_manager.cc:4584] TS 3ac3810becaa4d62810c6888b28175cb (127.1.44.129:41809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b93ade36c9e4fc1a45c98de00873a2b: Network error: Client connection negotiation failed: client connection to 127.1.44.129:41809: connect: Connection refused (error 111)
I20260924 00:18:02.622124 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21241
I20260924 00:18:02.626206 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 20888
2026-09-24T00:18:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (876 ms)
[----------] 6 tests from EncryptionPolicyTest (8086 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:02 dist-test-slave-cm7b krb5kdc[21412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:02 dist-test-slave-cm7b krb5kdc[21412](info): set up 2 sockets
Sep 24 00:18:02 dist-test-slave-cm7b krb5kdc[21412](info): commencing operation
krb5kdc: starting...
W20260924 00:18:03.650991 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:03 dist-test-slave-cm7b krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:03.766425 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:03.844203 21425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:03.844466 21425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:03.844506 21425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:03.845686 21425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:03.845738 21425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:03.845762 21425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:03.845783 21425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:03.845811 21425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:03.847226 21425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:03.847498 21425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:03.847707 21425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:03.852259 21431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:03.852351 21430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:03.852463 21425 server_base.cc:1034] running on GCE node
W20260924 00:18:03.852371 21433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:03.852823 21425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:03.853057 21425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:03.854192 21425 hybrid_clock.cc:648] HybridClock initialized: now 1790209083854173 us; error 35 us; skew 500 ppm
Sep 24 00:18:03 dist-test-slave-cm7b krb5kdc[21412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209083, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:03.856534 21425 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:03.856962 21425 webserver.cc:460] Webserver started at http://127.0.0.1:35069/ using document root <none> and password file <none>
I20260924 00:18:03.857173 21425 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:03.857242 21425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:03.857379 21425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:03.858361 21425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "612cb04712994609983a82c0d42bface"
format_stamp: "Formatted at 2026-09-24 00:18:03 on dist-test-slave-cm7b"
server_key: "c0f64e8b341268391a01bae0763b6cf6"
server_key_iv: "3d282c6679758e4d8290c173e4b064a1"
server_key_version: "encryptionkey@0"
I20260924 00:18:03.858691 21425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "612cb04712994609983a82c0d42bface"
format_stamp: "Formatted at 2026-09-24 00:18:03 on dist-test-slave-cm7b"
server_key: "c0f64e8b341268391a01bae0763b6cf6"
server_key_iv: "3d282c6679758e4d8290c173e4b064a1"
server_key_version: "encryptionkey@0"
I20260924 00:18:03.860010 21425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:03.860690 21440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:03.860845 21425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:03.860940 21425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "612cb04712994609983a82c0d42bface"
format_stamp: "Formatted at 2026-09-24 00:18:03 on dist-test-slave-cm7b"
server_key: "c0f64e8b341268391a01bae0763b6cf6"
server_key_iv: "3d282c6679758e4d8290c173e4b064a1"
server_key_version: "encryptionkey@0"
I20260924 00:18:03.861006 21425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:03.870177 21425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:03.872290 21425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:03.872439 21425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:03.876304 21425 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34215
I20260924 00:18:03.876335 21492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34215 every 8 connection(s)
I20260924 00:18:03.876648 21425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:03.877524 21493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:03.879690 21493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface: Bootstrap starting.
I20260924 00:18:03.880198 21493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:03.880334 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21425
I20260924 00:18:03.880446 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:03.880508 21493 log.cc:824] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:03.880648 1202 external_mini_cluster.cc:1428] Setting key eadc64a11e384213302b90ca5c1146dc
I20260924 00:18:03.881255 21493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface: No bootstrap required, opened a new log
I20260924 00:18:03.883388 21493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612cb04712994609983a82c0d42bface" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260924 00:18:03.883534 21493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:03.883581 21493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 612cb04712994609983a82c0d42bface, State: Initialized, Role: FOLLOWER
I20260924 00:18:03.883700 21493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612cb04712994609983a82c0d42bface" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260924 00:18:03.883771 21493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:03.883792 21493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:03.883859 21493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:03.884352 21493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612cb04712994609983a82c0d42bface" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260924 00:18:03.884434 21493 leader_election.cc:302] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 612cb04712994609983a82c0d42bface; no voters:
I20260924 00:18:03.884604 21493 leader_election.cc:288] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:03.884629 21498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:03.884734 21498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [term 1 LEADER]: Becoming Leader. State: Replica: 612cb04712994609983a82c0d42bface, State: Running, Role: LEADER
I20260924 00:18:03.884814 21498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612cb04712994609983a82c0d42bface" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } }
I20260924 00:18:03.884861 21493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:03.885097 21498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [sys.catalog]: SysCatalogTable state changed. Reason: New leader 612cb04712994609983a82c0d42bface. Latest consensus state: current_term: 1 leader_uuid: "612cb04712994609983a82c0d42bface" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612cb04712994609983a82c0d42bface" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } } }
I20260924 00:18:03.885162 21498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:03.885115 21499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "612cb04712994609983a82c0d42bface" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612cb04712994609983a82c0d42bface" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34215 } } }
I20260924 00:18:03.885360 21499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:03.885948 21502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:03.886425 21502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:18:03 dist-test-slave-cm7b krb5kdc[21412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 00:18:03.888579 21502 catalog_manager.cc:1347] Generated new cluster ID: f43b0816331c42afb4ddc1d30ee69622
I20260924 00:18:03.888625 21502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:03.891273 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:03.881104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38221 (local address 127.0.0.1:34215)
0924 00:18:03.881270 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:03.881279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:03.881310 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:18:03.881743 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:03.881750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:03.881796 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:03.881929 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:03.882545 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:03.883448 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:03.884330 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:03.884900 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:03.885492 (+ 592us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:03.887839 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:03.887844 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:03.887844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:03.887855 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:03.889639 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:03.890082 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:03.890084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:03.890085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:03.890178 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:03.890419 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:03.890420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:03.890420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:03.890680 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:03.890825 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:03.890968 (+ 143us) server_negotiation.cc:299] Negotiation successful
0924 00:18:03.891058 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260924 00:18:03.892211 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:03.900195 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:03.893578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47207 (local address 127.0.0.1:34215)
0924 00:18:03.893613 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:03.893614 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:03.893627 (+ 13us) server_negotiation.cc:407] Connection header received
0924 00:18:03.893695 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:03.893695 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:03.893704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:03.893762 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:03.894300 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:03.895020 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:03.895979 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:03.896285 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:03.896361 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:03.897562 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:03.897563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:03.897563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:03.897570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:03.899038 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:03.899430 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:03.899431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:03.899432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:03.899486 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:03.899693 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:03.899693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:03.899694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:03.899811 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:03.899902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:03.900046 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 00:18:03.900108 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:03.906845 21502 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:03.907557 21502 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:03.919663 21502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 612cb04712994609983a82c0d42bface: Generated new TSK 0
I20260924 00:18:03.919831 21502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:03.924053 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21425
W20260924 00:18:03.927999 21410 connection.cc:535] client connection to 127.0.0.1:34215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34215): Connection reset by peer (error 104)
2026-09-24T00:18:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:03 dist-test-slave-cm7b krb5kdc[21524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:03 dist-test-slave-cm7b krb5kdc[21524](info): set up 2 sockets
Sep 24 00:18:03 dist-test-slave-cm7b krb5kdc[21524](info): commencing operation
krb5kdc: starting...
W20260924 00:18:04.947134 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:05 dist-test-slave-cm7b krb5kdc[21524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:05.061965 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:05.138911 21537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:05.139106 21537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:05.139128 21537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:05.140216 21537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:05.140262 21537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:05.140277 21537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:05.140290 21537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:05.140302 21537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:05.141678 21537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:05.141856 21537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:05.142011 21537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:05.146700 21543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:05.146685 21545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:05.146664 21542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:05.146770 21537 server_base.cc:1034] running on GCE node
I20260924 00:18:05.147061 21537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:05.147374 21537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:05.148522 21537 hybrid_clock.cc:648] HybridClock initialized: now 1790209085148509 us; error 29 us; skew 500 ppm
Sep 24 00:18:05 dist-test-slave-cm7b krb5kdc[21524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209085, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:05.154191 21537 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:05.154613 21537 webserver.cc:460] Webserver started at http://127.0.0.1:40101/ using document root <none> and password file <none>
I20260924 00:18:05.154840 21537 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:05.154906 21537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:05.155046 21537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:05.156129 21537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "049724d8e88b4ef5b602b9ae499c3140"
format_stamp: "Formatted at 2026-09-24 00:18:05 on dist-test-slave-cm7b"
server_key: "9045e422c6cf72bac252a678b1d1356c"
server_key_iv: "a67769199e5ed05fd19b9eddc95aec23"
server_key_version: "encryptionkey@0"
I20260924 00:18:05.156482 21537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "049724d8e88b4ef5b602b9ae499c3140"
format_stamp: "Formatted at 2026-09-24 00:18:05 on dist-test-slave-cm7b"
server_key: "9045e422c6cf72bac252a678b1d1356c"
server_key_iv: "a67769199e5ed05fd19b9eddc95aec23"
server_key_version: "encryptionkey@0"
I20260924 00:18:05.157774 21537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 00:18:05.158555 21552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:05.158736 21537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:05.158839 21537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "049724d8e88b4ef5b602b9ae499c3140"
format_stamp: "Formatted at 2026-09-24 00:18:05 on dist-test-slave-cm7b"
server_key: "9045e422c6cf72bac252a678b1d1356c"
server_key_iv: "a67769199e5ed05fd19b9eddc95aec23"
server_key_version: "encryptionkey@0"
I20260924 00:18:05.158921 21537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:05.178453 21537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:05.180752 21537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:05.180902 21537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:05.184744 21537 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45127
I20260924 00:18:05.184774 21604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45127 every 8 connection(s)
I20260924 00:18:05.185103 21537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:05.185868 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21537
I20260924 00:18:05.185992 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:05.186115 21605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:05.186187 1202 external_mini_cluster.cc:1428] Setting key ba6fce08ece55890e8788c529bfb1f46
I20260924 00:18:05.189596 21605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140: Bootstrap starting.
I20260924 00:18:05.190229 21605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:05.190518 21605 log.cc:824] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:05.191336 21605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140: No bootstrap required, opened a new log
I20260924 00:18:05.192121 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:05.186715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36715 (local address 127.0.0.1:45127)
0924 00:18:05.186899 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:05.186910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:05.186942 (+ 32us) server_negotiation.cc:407] Connection header received
0924 00:18:05.187431 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:05.187441 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:05.187504 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:05.187703 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:05.187855 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:05.189222 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:05.189981 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:05.190608 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:05.190731 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:05.191461 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:05.191466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:05.191466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:05.191668 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:05.191704 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:05.191718 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:05.191807 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:05.191893 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 00:18:05.191918 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260924 00:18:05.193048 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:05.193450 21605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049724d8e88b4ef5b602b9ae499c3140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45127 } }
I20260924 00:18:05.193574 21605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:05.193598 21605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049724d8e88b4ef5b602b9ae499c3140, State: Initialized, Role: FOLLOWER
I20260924 00:18:05.193728 21605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049724d8e88b4ef5b602b9ae499c3140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45127 } }
I20260924 00:18:05.193804 21605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:05.193840 21605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:05.193888 21605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:05.194507 21605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049724d8e88b4ef5b602b9ae499c3140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45127 } }
I20260924 00:18:05.194620 21605 leader_election.cc:302] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 049724d8e88b4ef5b602b9ae499c3140; no voters:
I20260924 00:18:05.194782 21605 leader_election.cc:288] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:05.194813 21615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:05.194957 21615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [term 1 LEADER]: Becoming Leader. State: Replica: 049724d8e88b4ef5b602b9ae499c3140, State: Running, Role: LEADER
I20260924 00:18:05.195075 21605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:05.195091 21615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049724d8e88b4ef5b602b9ae499c3140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45127 } }
I20260924 00:18:05.195441 21615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 049724d8e88b4ef5b602b9ae499c3140. Latest consensus state: current_term: 1 leader_uuid: "049724d8e88b4ef5b602b9ae499c3140" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049724d8e88b4ef5b602b9ae499c3140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45127 } } }
I20260924 00:18:05.195454 21616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "049724d8e88b4ef5b602b9ae499c3140" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "049724d8e88b4ef5b602b9ae499c3140" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45127 } } }
I20260924 00:18:05.195577 21615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:05.195801 21616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:05.196359 21624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:05.196870 21624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:05.197813 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:05.194364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47647 (local address 127.0.0.1:45127)
0924 00:18:05.194404 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:05.194406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:05.194525 (+ 119us) server_negotiation.cc:407] Connection header received
0924 00:18:05.194605 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:05.194606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:05.194616 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:05.194687 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:05.194820 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:05.195475 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:05.196640 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:05.196925 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:05.196994 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:05.197614 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:05.197615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:05.197615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:05.197619 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:05.197630 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:05.197634 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:05.197691 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:05.197757 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 00:18:05.197765 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:05.200206 21624 catalog_manager.cc:1347] Generated new cluster ID: 0afb5fbf76e04b78aae411249a6490fb
I20260924 00:18:05.200263 21624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:05.210467 21624 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:05.211234 21624 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:05.219633 21624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 049724d8e88b4ef5b602b9ae499c3140: Generated new TSK 0
I20260924 00:18:05.219818 21624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:05.226488 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21537
W20260924 00:18:05.230170 21522 connection.cc:535] client connection to 127.0.0.1:45127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45127): Connection reset by peer (error 104)
2026-09-24T00:18:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:05 dist-test-slave-cm7b krb5kdc[21636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:05 dist-test-slave-cm7b krb5kdc[21636](info): set up 2 sockets
Sep 24 00:18:05 dist-test-slave-cm7b krb5kdc[21636](info): commencing operation
krb5kdc: starting...
W20260924 00:18:06.259110 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:06 dist-test-slave-cm7b krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:06.370091 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:06.449280 21649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:06.449478 21649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:06.449501 21649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:06.450634 21649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:06.450675 21649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:06.450690 21649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:06.450738 21649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:06.450759 21649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:06.452217 21649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:06.452471 21649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:06.452682 21649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:06.457377 21654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:06.457367 21657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:06.457451 21655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:06.457623 21649 server_base.cc:1034] running on GCE node
I20260924 00:18:06.457839 21649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:06.458062 21649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:06.459246 21649 hybrid_clock.cc:648] HybridClock initialized: now 1790209086459230 us; error 27 us; skew 500 ppm
Sep 24 00:18:06 dist-test-slave-cm7b krb5kdc[21636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209086, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:06.461483 21649 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:06.461925 21649 webserver.cc:460] Webserver started at http://127.0.0.1:35353/ using document root <none> and password file <none>
I20260924 00:18:06.462136 21649 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:06.462199 21649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:06.462337 21649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:06.463380 21649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "053fcba0bffb4eb685cd07f551412369"
format_stamp: "Formatted at 2026-09-24 00:18:06 on dist-test-slave-cm7b"
server_key: "7ea31f5dd4599b6bea56de26164b5715"
server_key_iv: "39bea3a0b7ce76e406ede44cc1f1280a"
server_key_version: "encryptionkey@0"
I20260924 00:18:06.463740 21649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "053fcba0bffb4eb685cd07f551412369"
format_stamp: "Formatted at 2026-09-24 00:18:06 on dist-test-slave-cm7b"
server_key: "7ea31f5dd4599b6bea56de26164b5715"
server_key_iv: "39bea3a0b7ce76e406ede44cc1f1280a"
server_key_version: "encryptionkey@0"
I20260924 00:18:06.464984 21649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:06.465658 21664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:06.465847 21649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:06.465950 21649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "053fcba0bffb4eb685cd07f551412369"
format_stamp: "Formatted at 2026-09-24 00:18:06 on dist-test-slave-cm7b"
server_key: "7ea31f5dd4599b6bea56de26164b5715"
server_key_iv: "39bea3a0b7ce76e406ede44cc1f1280a"
server_key_version: "encryptionkey@0"
I20260924 00:18:06.466018 21649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:06.469224 21649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:06.471114 21649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:06.471283 21649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:06.475047 21649 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43001
I20260924 00:18:06.475080 21716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43001 every 8 connection(s)
I20260924 00:18:06.475415 21649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:06.476307 21717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:06.478578 21717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369: Bootstrap starting.
I20260924 00:18:06.479112 21717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:06.479414 21717 log.cc:824] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:06.480018 21717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369: No bootstrap required, opened a new log
I20260924 00:18:06.481511 21717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053fcba0bffb4eb685cd07f551412369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43001 } }
I20260924 00:18:06.481637 21717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:06.481660 21717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053fcba0bffb4eb685cd07f551412369, State: Initialized, Role: FOLLOWER
I20260924 00:18:06.481781 21717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053fcba0bffb4eb685cd07f551412369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43001 } }
I20260924 00:18:06.481851 21717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:06.481873 21717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:06.481906 21717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:06.482360 21717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053fcba0bffb4eb685cd07f551412369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43001 } }
I20260924 00:18:06.482437 21717 leader_election.cc:302] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053fcba0bffb4eb685cd07f551412369; no voters:
I20260924 00:18:06.482551 21717 leader_election.cc:288] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:06.482622 21720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:06.482805 21717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:06.482956 21720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [term 1 LEADER]: Becoming Leader. State: Replica: 053fcba0bffb4eb685cd07f551412369, State: Running, Role: LEADER
I20260924 00:18:06.483047 21720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053fcba0bffb4eb685cd07f551412369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43001 } }
I20260924 00:18:06.483479 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053fcba0bffb4eb685cd07f551412369. Latest consensus state: current_term: 1 leader_uuid: "053fcba0bffb4eb685cd07f551412369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053fcba0bffb4eb685cd07f551412369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43001 } } }
I20260924 00:18:06.483583 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:06.483498 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "053fcba0bffb4eb685cd07f551412369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053fcba0bffb4eb685cd07f551412369" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43001 } } }
I20260924 00:18:06.483788 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:06.483810 21729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:06.484103 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21649
I20260924 00:18:06.484222 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:06.484272 21729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:06.484416 1202 external_mini_cluster.cc:1428] Setting key 54893577fe73b141c07cf40c3c617d3f
I20260924 00:18:06.486507 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:06.484976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59633 (local address 127.0.0.1:43001)
0924 00:18:06.485116 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:06.485125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:06.485156 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:18:06.485263 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:06.485269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:06.485299 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:06.485427 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:06.486038 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:06.486041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:06.486041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:06.486045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:06.486083 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:06.486097 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:06.486172 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:06.486205 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 00:18:06.486330 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260924 00:18:06.487390 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:06.487619 21729 catalog_manager.cc:1347] Generated new cluster ID: 799fabdf1e854f5bb9e14ddad1eb09cb
I20260924 00:18:06.487665 21729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:06.489621 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:06.488804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58767 (local address 127.0.0.1:43001)
0924 00:18:06.488839 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:06.488841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:06.488894 (+ 53us) server_negotiation.cc:407] Connection header received
0924 00:18:06.488956 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:06.488956 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:06.488960 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:06.489009 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:06.489464 (+ 455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:06.489465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:06.489465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:06.489468 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:06.489480 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:06.489483 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:06.489526 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:06.489571 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 00:18:06.489581 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:18:06.498106 21729 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:06.498263 21729 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:06.503490 21729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053fcba0bffb4eb685cd07f551412369: Generated new TSK 0
I20260924 00:18:06.503643 21729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:06.508824 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21649
2026-09-24T00:18:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:06 dist-test-slave-cm7b krb5kdc[21748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:06 dist-test-slave-cm7b krb5kdc[21748](info): set up 2 sockets
Sep 24 00:18:06 dist-test-slave-cm7b krb5kdc[21748](info): commencing operation
krb5kdc: starting...
W20260924 00:18:07.535460 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:07 dist-test-slave-cm7b krb5kdc[21748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:07.650379 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:07.733089 21761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:07.733333 21761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:07.733371 21761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:07.734576 21761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:07.734635 21761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:07.734666 21761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:07.734696 21761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:07.734719 21761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:07.736188 21761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:07.736451 21761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:07.736670 21761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:07.741340 21766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:07.741394 21767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:07.741508 21761 server_base.cc:1034] running on GCE node
W20260924 00:18:07.741340 21769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:07.741779 21761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:07.742069 21761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:07.743301 21761 hybrid_clock.cc:648] HybridClock initialized: now 1790209087743289 us; error 34 us; skew 500 ppm
Sep 24 00:18:07 dist-test-slave-cm7b krb5kdc[21748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209087, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:07.745689 21761 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:07.746107 21761 webserver.cc:460] Webserver started at http://127.0.0.1:38867/ using document root <none> and password file <none>
I20260924 00:18:07.746366 21761 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:07.746500 21761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:07.746667 21761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:07.747776 21761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "ab0d1f86135e4f299d6bd92a05ca9131"
format_stamp: "Formatted at 2026-09-24 00:18:07 on dist-test-slave-cm7b"
server_key: "9b2a27ef27df2b11474d59a4de875b67"
server_key_iv: "c5a1d4d9d4ae852d7a1dd520a6599dd8"
server_key_version: "encryptionkey@0"
I20260924 00:18:07.748123 21761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "ab0d1f86135e4f299d6bd92a05ca9131"
format_stamp: "Formatted at 2026-09-24 00:18:07 on dist-test-slave-cm7b"
server_key: "9b2a27ef27df2b11474d59a4de875b67"
server_key_iv: "c5a1d4d9d4ae852d7a1dd520a6599dd8"
server_key_version: "encryptionkey@0"
I20260924 00:18:07.749469 21761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:07.750212 21776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:07.750368 21761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:07.750465 21761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "ab0d1f86135e4f299d6bd92a05ca9131"
format_stamp: "Formatted at 2026-09-24 00:18:07 on dist-test-slave-cm7b"
server_key: "9b2a27ef27df2b11474d59a4de875b67"
server_key_iv: "c5a1d4d9d4ae852d7a1dd520a6599dd8"
server_key_version: "encryptionkey@0"
I20260924 00:18:07.750533 21761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:07.759075 21761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:07.760818 21761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:07.760967 21761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:07.764768 21761 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36059
I20260924 00:18:07.764796 21828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36059 every 8 connection(s)
I20260924 00:18:07.765120 21761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:07.766057 21829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:07.768286 21829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131: Bootstrap starting.
I20260924 00:18:07.768849 21829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:07.769129 21829 log.cc:824] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:07.769776 21829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131: No bootstrap required, opened a new log
I20260924 00:18:07.771409 21829 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36059 } }
I20260924 00:18:07.771541 21829 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:07.771564 21829 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0d1f86135e4f299d6bd92a05ca9131, State: Initialized, Role: FOLLOWER
I20260924 00:18:07.771713 21829 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36059 } }
I20260924 00:18:07.771772 21829 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:07.771793 21829 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:07.771857 21829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:07.772385 21829 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36059 } }
I20260924 00:18:07.772491 21829 leader_election.cc:302] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab0d1f86135e4f299d6bd92a05ca9131; no voters:
I20260924 00:18:07.772694 21829 leader_election.cc:288] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:07.772727 21832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:07.772948 21829 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:07.772933 21832 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [term 1 LEADER]: Becoming Leader. State: Replica: ab0d1f86135e4f299d6bd92a05ca9131, State: Running, Role: LEADER
I20260924 00:18:07.773144 21832 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36059 } }
I20260924 00:18:07.773763 21834 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab0d1f86135e4f299d6bd92a05ca9131. Latest consensus state: current_term: 1 leader_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36059 } } }
I20260924 00:18:07.773874 21834 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:07.773960 21833 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0d1f86135e4f299d6bd92a05ca9131" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36059 } } }
I20260924 00:18:07.774045 21833 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:07.774133 21842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:07.774662 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21761
I20260924 00:18:07.774705 21842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:07.774796 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:07.774959 1202 external_mini_cluster.cc:1428] Setting key b1000dc50df5013b6d67738ef4ad714d
I20260924 00:18:07.776461 21842 catalog_manager.cc:1347] Generated new cluster ID: b8026cf42082421892906305dc7a8688
I20260924 00:18:07.776512 21842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:18:07 dist-test-slave-cm7b krb5kdc[21748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 00:18:07.786445 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:07.775434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43671 (local address 127.0.0.1:36059)
0924 00:18:07.775591 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:07.775601 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:07.775642 (+ 41us) server_negotiation.cc:407] Connection header received
0924 00:18:07.775760 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:07.775766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:07.775821 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:07.775968 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:07.776796 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:07.778158 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:07.779290 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:07.779903 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:07.779997 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:07.782752 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:07.782757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:07.782757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:07.782771 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:07.784811 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:07.785186 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:07.785189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:07.785189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:07.785314 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:07.785507 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:07.785508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:07.785508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:07.785836 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:07.786000 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:07.786072 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 00:18:07.786210 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260924 00:18:07.787472 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:07.794394 21842 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:07.794941 21842 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:07.796065 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:07.788652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41221 (local address 127.0.0.1:36059)
0924 00:18:07.788709 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:07.788711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:07.788916 (+ 205us) server_negotiation.cc:407] Connection header received
0924 00:18:07.788974 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:07.788975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:07.788994 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:07.789040 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:07.789668 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:07.790428 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:07.791463 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:07.791766 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:07.791825 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:07.793201 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:07.793202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:07.793202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:07.793209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:07.794877 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:07.795288 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:07.795289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:07.795290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:07.795350 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:07.795548 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:07.795549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:07.795549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:07.795676 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:07.795773 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:07.795881 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 00:18:07.795974 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 00:18:07.807888 21842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab0d1f86135e4f299d6bd92a05ca9131: Generated new TSK 0
I20260924 00:18:07.808127 21842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:07.810525 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21761
2026-09-24T00:18:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:07 dist-test-slave-cm7b krb5kdc[21860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:07 dist-test-slave-cm7b krb5kdc[21860](info): set up 2 sockets
Sep 24 00:18:07 dist-test-slave-cm7b krb5kdc[21860](info): commencing operation
krb5kdc: starting...
W20260924 00:18:08.839166 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:08 dist-test-slave-cm7b krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209088, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:08.954654 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:09.034364 21873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:09.034590 21873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:09.034631 21873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:09.035805 21873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:09.035866 21873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:09.035889 21873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:09.035919 21873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:09.035948 21873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:09.037370 21873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:09.037638 21873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:09.037858 21873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:09.042446 21879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:09.042495 21878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:09.042448 21881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:09.042616 21873 server_base.cc:1034] running on GCE node
I20260924 00:18:09.042910 21873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:09.043146 21873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:09.044337 21873 hybrid_clock.cc:648] HybridClock initialized: now 1790209089044324 us; error 28 us; skew 500 ppm
Sep 24 00:18:09 dist-test-slave-cm7b krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209089, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:09.046885 21873 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:09.047421 21873 webserver.cc:460] Webserver started at http://127.0.0.1:42365/ using document root <none> and password file <none>
I20260924 00:18:09.047652 21873 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:09.047713 21873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:09.047847 21873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:09.048857 21873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "b0c38640d5e54d129e1f163af5abcd16"
format_stamp: "Formatted at 2026-09-24 00:18:09 on dist-test-slave-cm7b"
server_key: "f85722a0f9f40c416a95fec629fac7f7"
server_key_iv: "03f2764add9f1205f8b4885bec47e5f8"
server_key_version: "encryptionkey@0"
I20260924 00:18:09.049208 21873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "b0c38640d5e54d129e1f163af5abcd16"
format_stamp: "Formatted at 2026-09-24 00:18:09 on dist-test-slave-cm7b"
server_key: "f85722a0f9f40c416a95fec629fac7f7"
server_key_iv: "03f2764add9f1205f8b4885bec47e5f8"
server_key_version: "encryptionkey@0"
I20260924 00:18:09.050509 21873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:18:09.051296 21888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:09.051472 21873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:09.051566 21873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "b0c38640d5e54d129e1f163af5abcd16"
format_stamp: "Formatted at 2026-09-24 00:18:09 on dist-test-slave-cm7b"
server_key: "f85722a0f9f40c416a95fec629fac7f7"
server_key_iv: "03f2764add9f1205f8b4885bec47e5f8"
server_key_version: "encryptionkey@0"
I20260924 00:18:09.051648 21873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:09.063113 21873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:09.064896 21873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:09.065054 21873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:09.069067 21873 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41207
I20260924 00:18:09.069116 21940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41207 every 8 connection(s)
I20260924 00:18:09.069473 21873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:09.070401 21941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:09.072721 21941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16: Bootstrap starting.
I20260924 00:18:09.073307 21941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:09.073578 21941 log.cc:824] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:09.074302 21941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16: No bootstrap required, opened a new log
I20260924 00:18:09.075954 21941 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0c38640d5e54d129e1f163af5abcd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 00:18:09.076088 21941 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:09.076110 21941 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0c38640d5e54d129e1f163af5abcd16, State: Initialized, Role: FOLLOWER
I20260924 00:18:09.076270 21941 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0c38640d5e54d129e1f163af5abcd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 00:18:09.076339 21941 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:09.076382 21941 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:09.076447 21941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:09.076958 21941 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0c38640d5e54d129e1f163af5abcd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 00:18:09.077040 21941 leader_election.cc:302] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0c38640d5e54d129e1f163af5abcd16; no voters:
I20260924 00:18:09.077212 21941 leader_election.cc:288] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:09.077280 21944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:09.077450 21944 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [term 1 LEADER]: Becoming Leader. State: Replica: b0c38640d5e54d129e1f163af5abcd16, State: Running, Role: LEADER
I20260924 00:18:09.077461 21941 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:09.077574 21944 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0c38640d5e54d129e1f163af5abcd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 00:18:09.077956 21946 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0c38640d5e54d129e1f163af5abcd16. Latest consensus state: current_term: 1 leader_uuid: "b0c38640d5e54d129e1f163af5abcd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0c38640d5e54d129e1f163af5abcd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } } }
I20260924 00:18:09.078064 21946 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:09.077953 21945 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0c38640d5e54d129e1f163af5abcd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0c38640d5e54d129e1f163af5abcd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } } }
I20260924 00:18:09.078114 21945 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:09.078258 21948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:09.078751 21948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:09.079071 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21873
I20260924 00:18:09.079212 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:09.079391 1202 external_mini_cluster.cc:1428] Setting key d27d088ad3de266b40bfd4ec03d0eddd
I20260924 00:18:09.080451 21948 catalog_manager.cc:1347] Generated new cluster ID: fe3e8e1e28384f89b94f85dff724dce2
I20260924 00:18:09.080510 21948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:09.084548 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:09.079838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40917 (local address 127.0.0.1:41207)
0924 00:18:09.080002 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:09.080013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:09.080139 (+ 126us) server_negotiation.cc:407] Connection header received
0924 00:18:09.080261 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:09.080268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:09.080327 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:09.080555 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:09.080926 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:09.081867 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:09.082637 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:09.083064 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:09.083174 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:09.083903 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:09.083907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:09.083907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:09.084079 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:09.084110 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:09.084120 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:09.084189 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:09.084273 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 00:18:09.084339 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260924 00:18:09.085358 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:09.089359 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:09.086631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57229 (local address 127.0.0.1:41207)
0924 00:18:09.086667 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:09.086669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:09.086795 (+ 126us) server_negotiation.cc:407] Connection header received
0924 00:18:09.086863 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:09.086863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:09.086873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:09.086917 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:09.087013 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:09.087535 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:09.088212 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:09.088426 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:09.088498 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:09.089128 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:09.089129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:09.089129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:09.089135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:09.089147 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:09.089151 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:09.089198 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:09.089243 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 00:18:09.089265 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:09.103457 21948 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:09.103978 21948 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:09.117796 21948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0c38640d5e54d129e1f163af5abcd16: Generated new TSK 0
I20260924 00:18:09.118014 21948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:09.125928 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21873
2026-09-24T00:18:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:09 dist-test-slave-cm7b krb5kdc[21972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:09 dist-test-slave-cm7b krb5kdc[21972](info): set up 2 sockets
Sep 24 00:18:09 dist-test-slave-cm7b krb5kdc[21972](info): commencing operation
krb5kdc: starting...
W20260924 00:18:10.163105 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:10 dist-test-slave-cm7b krb5kdc[21972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:10.276117 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:10.355412 21985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:10.355639 21985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:10.355676 21985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:10.356824 21985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:10.356889 21985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:10.356915 21985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:10.356935 21985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:10.356967 21985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:10.358392 21985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:10.358659 21985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:10.358877 21985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:10.363526 21990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:10.363528 21993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:10.363777 21985 server_base.cc:1034] running on GCE node
W20260924 00:18:10.363528 21991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:10.364099 21985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:10.364385 21985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:10.365542 21985 hybrid_clock.cc:648] HybridClock initialized: now 1790209090365522 us; error 29 us; skew 500 ppm
Sep 24 00:18:10 dist-test-slave-cm7b krb5kdc[21972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209090, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:10.373718 21985 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:10.374154 21985 webserver.cc:460] Webserver started at http://127.0.0.1:43759/ using document root <none> and password file <none>
I20260924 00:18:10.374378 21985 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:10.374423 21985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:10.374577 21985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:10.375732 21985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "b5e112101d6f423e8b627419e771d08c"
format_stamp: "Formatted at 2026-09-24 00:18:10 on dist-test-slave-cm7b"
server_key: "81340a82605443e50a3716f69d4b99c8"
server_key_iv: "52b27dd5f1d9594d4b1c243033dd741a"
server_key_version: "encryptionkey@0"
I20260924 00:18:10.376080 21985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "b5e112101d6f423e8b627419e771d08c"
format_stamp: "Formatted at 2026-09-24 00:18:10 on dist-test-slave-cm7b"
server_key: "81340a82605443e50a3716f69d4b99c8"
server_key_iv: "52b27dd5f1d9594d4b1c243033dd741a"
server_key_version: "encryptionkey@0"
I20260924 00:18:10.377398 21985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:10.378136 22000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:10.378394 21985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:10.378526 21985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "b5e112101d6f423e8b627419e771d08c"
format_stamp: "Formatted at 2026-09-24 00:18:10 on dist-test-slave-cm7b"
server_key: "81340a82605443e50a3716f69d4b99c8"
server_key_iv: "52b27dd5f1d9594d4b1c243033dd741a"
server_key_version: "encryptionkey@0"
I20260924 00:18:10.378604 21985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:10.382149 21985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:10.383949 21985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:10.384059 21985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:10.387784 21985 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39883
I20260924 00:18:10.387827 22052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39883 every 8 connection(s)
I20260924 00:18:10.388140 21985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:10.389096 22053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:10.390525 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 21985
I20260924 00:18:10.390642 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:10.390803 1202 external_mini_cluster.cc:1428] Setting key ab1e20a84a7e69cf201d3cdcb761b3e2
I20260924 00:18:10.391520 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c: Bootstrap starting.
I20260924 00:18:10.392092 22053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:10.392433 22053 log.cc:824] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:10.393209 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c: No bootstrap required, opened a new log
I20260924 00:18:10.393366 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:10.391352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38979 (local address 127.0.0.1:39883)
0924 00:18:10.391521 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:10.391528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:10.391552 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:18:10.392003 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:10.392009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:10.392034 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:10.392166 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:10.392850 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:10.392854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:10.392855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:10.392860 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:10.392903 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:10.392917 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:10.392995 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:10.393072 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 00:18:10.393120 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260924 00:18:10.394508 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:10.395704 22053 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e112101d6f423e8b627419e771d08c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39883 } }
I20260924 00:18:10.395900 22053 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:10.395998 22053 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e112101d6f423e8b627419e771d08c, State: Initialized, Role: FOLLOWER
I20260924 00:18:10.396149 22053 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e112101d6f423e8b627419e771d08c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39883 } }
I20260924 00:18:10.396299 22053 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:10.396349 22053 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:10.396407 22053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:10.396906 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:10.395979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54287 (local address 127.0.0.1:39883)
0924 00:18:10.396015 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:10.396017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:10.396090 (+ 73us) server_negotiation.cc:407] Connection header received
0924 00:18:10.396188 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:10.396189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:10.396193 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:10.396243 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:10.396744 (+ 501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:10.396745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:10.396745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:10.396748 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:10.396760 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:10.396763 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:10.396807 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:10.396858 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 00:18:10.396868 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:18:10.397153 22053 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e112101d6f423e8b627419e771d08c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39883 } }
I20260924 00:18:10.397281 22053 leader_election.cc:302] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5e112101d6f423e8b627419e771d08c; no voters:
I20260924 00:18:10.397451 22053 leader_election.cc:288] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:10.397496 22063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:10.397650 22063 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [term 1 LEADER]: Becoming Leader. State: Replica: b5e112101d6f423e8b627419e771d08c, State: Running, Role: LEADER
I20260924 00:18:10.397679 22053 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:10.397789 22063 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e112101d6f423e8b627419e771d08c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39883 } }
I20260924 00:18:10.398119 22064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5e112101d6f423e8b627419e771d08c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e112101d6f423e8b627419e771d08c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39883 } } }
I20260924 00:18:10.398124 22065 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e112101d6f423e8b627419e771d08c. Latest consensus state: current_term: 1 leader_uuid: "b5e112101d6f423e8b627419e771d08c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e112101d6f423e8b627419e771d08c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39883 } } }
I20260924 00:18:10.398263 22064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:10.398270 22065 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c [sys.catalog]: This master's current role is: LEADER
W20260924 00:18:10.399538 22079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:10.399581 22079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:10.399638 22071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:10.399780 22071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:10.401008 22071 catalog_manager.cc:1347] Generated new cluster ID: 166a3438090f4353acd81d6deb201852
I20260924 00:18:10.401063 22071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:10.426010 22071 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:10.426234 22071 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:10.431901 22071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5e112101d6f423e8b627419e771d08c: Generated new TSK 0
I20260924 00:18:10.432077 22071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:10.439363 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 21985
2026-09-24T00:18:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:10 dist-test-slave-cm7b krb5kdc[22084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:10 dist-test-slave-cm7b krb5kdc[22084](info): set up 2 sockets
Sep 24 00:18:10 dist-test-slave-cm7b krb5kdc[22084](info): commencing operation
krb5kdc: starting...
W20260924 00:18:11.459141 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:11 dist-test-slave-cm7b krb5kdc[22084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:11.577510 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39035
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:11.656277 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:11.656507 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:11.656540 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:11.657632 22097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:11.657702 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:11.657727 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:11.657756 22097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:11.657786 22097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:11.659240 22097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:11.659512 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:11.659734 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:11.664399 22103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:11.664480 22105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:11.664479 22102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:11.664448 22097 server_base.cc:1034] running on GCE node
I20260924 00:18:11.664825 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:11.665112 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:11.666276 22097 hybrid_clock.cc:648] HybridClock initialized: now 1790209091666262 us; error 26 us; skew 500 ppm
Sep 24 00:18:11 dist-test-slave-cm7b krb5kdc[22084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209091, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:11.668730 22097 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:11.669126 22097 webserver.cc:460] Webserver started at http://127.0.0.1:44721/ using document root <none> and password file <none>
I20260924 00:18:11.669318 22097 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:11.669359 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:11.669443 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:11.670395 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "705f8dfc1881478f81fe583403a46199"
format_stamp: "Formatted at 2026-09-24 00:18:11 on dist-test-slave-cm7b"
server_key: "73965610d4219c11071d415315cfe0dd"
server_key_iv: "de5c726e526386e4a461fda1935f97d4"
server_key_version: "encryptionkey@0"
I20260924 00:18:11.670679 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "705f8dfc1881478f81fe583403a46199"
format_stamp: "Formatted at 2026-09-24 00:18:11 on dist-test-slave-cm7b"
server_key: "73965610d4219c11071d415315cfe0dd"
server_key_iv: "de5c726e526386e4a461fda1935f97d4"
server_key_version: "encryptionkey@0"
I20260924 00:18:11.671984 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:11.672644 22112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:11.672813 22097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:11.672894 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "705f8dfc1881478f81fe583403a46199"
format_stamp: "Formatted at 2026-09-24 00:18:11 on dist-test-slave-cm7b"
server_key: "73965610d4219c11071d415315cfe0dd"
server_key_iv: "de5c726e526386e4a461fda1935f97d4"
server_key_version: "encryptionkey@0"
I20260924 00:18:11.672943 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:11.689561 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:11.691437 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:11.691551 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:11.695228 22097 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39035
I20260924 00:18:11.695266 22164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39035 every 8 connection(s)
I20260924 00:18:11.695685 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:11.697058 22165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:11.700747 22165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199: Bootstrap starting.
I20260924 00:18:11.701344 22165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:11.701673 22165 log.cc:824] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:11.701938 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22097
I20260924 00:18:11.702049 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:11.702253 1202 external_mini_cluster.cc:1428] Setting key 59bc7c3afe0bb63b2d376b793fe5caf7
I20260924 00:18:11.702843 22165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199: No bootstrap required, opened a new log
I20260924 00:18:11.705026 22165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705f8dfc1881478f81fe583403a46199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39035 } }
I20260924 00:18:11.705232 22165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:11.705279 22165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 705f8dfc1881478f81fe583403a46199, State: Initialized, Role: FOLLOWER
I20260924 00:18:11.705420 22165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705f8dfc1881478f81fe583403a46199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39035 } }
I20260924 00:18:11.705514 22165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:11.705547 22165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:11.705593 22165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:11.706274 22165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705f8dfc1881478f81fe583403a46199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39035 } }
I20260924 00:18:11.706387 22165 leader_election.cc:302] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 705f8dfc1881478f81fe583403a46199; no voters:
I20260924 00:18:11.706528 22165 leader_election.cc:288] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:11.706580 22170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:11.706728 22170 raft_consensus.cc:686] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [term 1 LEADER]: Becoming Leader. State: Replica: 705f8dfc1881478f81fe583403a46199, State: Running, Role: LEADER
I20260924 00:18:11.706748 22165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:11.706811 22170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705f8dfc1881478f81fe583403a46199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39035 } }
I20260924 00:18:11.707252 22171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "705f8dfc1881478f81fe583403a46199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705f8dfc1881478f81fe583403a46199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39035 } } }
I20260924 00:18:11.707329 22171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:11.707527 22172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 705f8dfc1881478f81fe583403a46199. Latest consensus state: current_term: 1 leader_uuid: "705f8dfc1881478f81fe583403a46199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705f8dfc1881478f81fe583403a46199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39035 } } }
I20260924 00:18:11.707842 22172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:11.707970 22179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:11.708478 22179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 00:18:11 dist-test-slave-cm7b krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 00:18:11.711004 22179 catalog_manager.cc:1347] Generated new cluster ID: ee435d4ec94a4f69a5087eb178d44e80
I20260924 00:18:11.711066 22179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:11.714402 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:11.702746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:59877 (local address 127.0.0.1:39035)
0924 00:18:11.702904 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:11.702914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:11.703283 (+ 369us) server_negotiation.cc:407] Connection header received
0924 00:18:11.704265 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:11.704274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:11.704342 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:11.704492 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:11.705122 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:11.706828 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:11.708228 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:11.708638 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:11.708760 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:11.711171 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:11.711175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:11.711175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:11.711186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:11.712866 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:11.713270 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:11.713273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:11.713274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:11.713359 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:11.713587 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:11.713588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:11.713589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:11.713815 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:11.713990 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:11.714111 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 00:18:11.714210 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260924 00:18:11.715274 22179 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:11.715389 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:11.715919 22179 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:11.718693 22179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 705f8dfc1881478f81fe583403a46199: Generated new TSK 0
I20260924 00:18:11.718855 22179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:11.722793 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:11.716726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:34971 (local address 127.0.0.1:39035)
0924 00:18:11.716764 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:11.716766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:11.716887 (+ 121us) server_negotiation.cc:407] Connection header received
0924 00:18:11.716957 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:11.716958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:11.716968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:11.717021 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:11.717518 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:11.718016 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:11.718983 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:11.719178 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:11.719295 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:11.720429 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:11.720431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:11.720431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:11.720438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:11.721658 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:11.722014 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:11.722015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:11.722015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:11.722070 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:11.722305 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:11.722306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:11.722306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:11.722409 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:11.722515 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:11.722632 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 00:18:11.722691 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:11.724686 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22097
W20260924 00:18:11.728543 22082 connection.cc:535] client connection to 127.0.0.1:39035 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39035): Connection reset by peer (error 104)
2026-09-24T00:18:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:11 dist-test-slave-cm7b krb5kdc[22196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:11 dist-test-slave-cm7b krb5kdc[22196](info): set up 2 sockets
Sep 24 00:18:11 dist-test-slave-cm7b krb5kdc[22196](info): commencing operation
krb5kdc: starting...
W20260924 00:18:12.751072 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:12 dist-test-slave-cm7b krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:12.861835 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:12.939535 22209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:12.939767 22209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:12.939805 22209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:12.940953 22209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:12.941020 22209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:12.941044 22209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:12.941066 22209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:12.941095 22209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:12.942551 22209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:12.942822 22209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:12.943041 22209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:12.947572 22217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:12.947598 22214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:12.947584 22215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:12.947903 22209 server_base.cc:1034] running on GCE node
I20260924 00:18:12.948112 22209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:12.948334 22209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:12.949469 22209 hybrid_clock.cc:648] HybridClock initialized: now 1790209092949456 us; error 29 us; skew 500 ppm
Sep 24 00:18:12 dist-test-slave-cm7b krb5kdc[22196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209092, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:12.951805 22209 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:12.952236 22209 webserver.cc:460] Webserver started at http://127.0.0.1:38557/ using document root <none> and password file <none>
I20260924 00:18:12.952409 22209 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:12.952455 22209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:12.952540 22209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:12.953501 22209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "7ab829e8c710403a8968bbcea38cdbef"
format_stamp: "Formatted at 2026-09-24 00:18:12 on dist-test-slave-cm7b"
server_key: "36bd542dbfde2203a14f11bf0bb99998"
server_key_iv: "93e5e396592fcfb975f8d5d8be23561e"
server_key_version: "encryptionkey@0"
I20260924 00:18:12.953788 22209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "7ab829e8c710403a8968bbcea38cdbef"
format_stamp: "Formatted at 2026-09-24 00:18:12 on dist-test-slave-cm7b"
server_key: "36bd542dbfde2203a14f11bf0bb99998"
server_key_iv: "93e5e396592fcfb975f8d5d8be23561e"
server_key_version: "encryptionkey@0"
I20260924 00:18:12.954978 22209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:12.955708 22224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:12.955873 22209 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:12.955981 22209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "7ab829e8c710403a8968bbcea38cdbef"
format_stamp: "Formatted at 2026-09-24 00:18:12 on dist-test-slave-cm7b"
server_key: "36bd542dbfde2203a14f11bf0bb99998"
server_key_iv: "93e5e396592fcfb975f8d5d8be23561e"
server_key_version: "encryptionkey@0"
I20260924 00:18:12.956053 22209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:12.970638 22209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:12.972441 22209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:12.972592 22209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:12.976667 22209 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32897
I20260924 00:18:12.976699 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32897 every 8 connection(s)
I20260924 00:18:12.977022 22209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:12.977921 22277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:12.980115 22277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef: Bootstrap starting.
I20260924 00:18:12.980690 22277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:12.980954 22277 log.cc:824] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:12.981554 22277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef: No bootstrap required, opened a new log
I20260924 00:18:12.983004 22277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab829e8c710403a8968bbcea38cdbef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260924 00:18:12.983132 22277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:12.983155 22277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab829e8c710403a8968bbcea38cdbef, State: Initialized, Role: FOLLOWER
I20260924 00:18:12.983314 22277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab829e8c710403a8968bbcea38cdbef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260924 00:18:12.983376 22277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:12.983397 22277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:12.983423 22277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:12.983875 22277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab829e8c710403a8968bbcea38cdbef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260924 00:18:12.983955 22277 leader_election.cc:302] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ab829e8c710403a8968bbcea38cdbef; no voters:
I20260924 00:18:12.984067 22277 leader_election.cc:288] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:12.984140 22280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:12.984294 22280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [term 1 LEADER]: Becoming Leader. State: Replica: 7ab829e8c710403a8968bbcea38cdbef, State: Running, Role: LEADER
I20260924 00:18:12.984330 22277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:12.984382 22280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab829e8c710403a8968bbcea38cdbef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } }
I20260924 00:18:12.984794 22282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ab829e8c710403a8968bbcea38cdbef. Latest consensus state: current_term: 1 leader_uuid: "7ab829e8c710403a8968bbcea38cdbef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab829e8c710403a8968bbcea38cdbef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260924 00:18:12.984880 22282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:12.985006 22281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ab829e8c710403a8968bbcea38cdbef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ab829e8c710403a8968bbcea38cdbef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32897 } } }
I20260924 00:18:12.985100 22281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:12.985117 22289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:12.985603 22289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:12.985877 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22209
I20260924 00:18:12.985981 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:12.986151 1202 external_mini_cluster.cc:1428] Setting key 1c977e0795f408298b653b952193b3b2
I20260924 00:18:12.987149 22289 catalog_manager.cc:1347] Generated new cluster ID: 682f25081c7244d387445104431c458b
I20260924 00:18:12.987221 22289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:12.992051 22299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:12.986597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:56791 (local address 127.0.0.1:32897)
0924 00:18:12.986769 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:12.986778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:12.986811 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:18:12.986920 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:12.986926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:12.986980 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:12.987121 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:12.987501 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:12.988839 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:12.989966 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:12.990539 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:12.990635 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:12.991376 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:12.991379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:12.991380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:12.991570 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:12.991605 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:12.991619 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:12.991693 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:12.991822 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 00:18:12.991845 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260924 00:18:12.992893 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:12.997712 22299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:12.994328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:38389 (local address 127.0.0.1:32897)
0924 00:18:12.994369 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:12.994370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:12.994516 (+ 146us) server_negotiation.cc:407] Connection header received
0924 00:18:12.994573 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:12.994574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:12.994583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:12.994636 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:12.994767 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:12.995474 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:12.996447 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:12.996749 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:12.996808 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:12.997465 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:12.997466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:12.997466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:12.997470 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:12.997483 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:12.997486 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:12.997538 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:12.997632 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 00:18:12.997644 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 00:18:13.000649 22289 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:13.001132 22289 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:13.010658 22289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ab829e8c710403a8968bbcea38cdbef: Generated new TSK 0
I20260924 00:18:13.010887 22289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:13.018965 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22209
W20260924 00:18:13.023324 22194 connection.cc:535] client connection to 127.0.0.1:32897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32897): Connection reset by peer (error 104)
2026-09-24T00:18:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:13 dist-test-slave-cm7b krb5kdc[22308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:13 dist-test-slave-cm7b krb5kdc[22308](info): set up 2 sockets
Sep 24 00:18:13 dist-test-slave-cm7b krb5kdc[22308](info): commencing operation
krb5kdc: starting...
W20260924 00:18:14.055119 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:14 dist-test-slave-cm7b krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:14.168262 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:14.248135 22321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:14.248373 22321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:14.248410 22321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:14.249567 22321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:14.249635 22321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:14.249661 22321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:14.249694 22321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:14.249718 22321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:14.251170 22321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:14.251495 22321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:14.251718 22321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:14.256440 22329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:14.256464 22327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:14.256448 22326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:14.256697 22321 server_base.cc:1034] running on GCE node
I20260924 00:18:14.256949 22321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:14.257241 22321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:14.258404 22321 hybrid_clock.cc:648] HybridClock initialized: now 1790209094258392 us; error 43 us; skew 500 ppm
Sep 24 00:18:14 dist-test-slave-cm7b krb5kdc[22308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209094, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:14.260847 22321 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:14.261303 22321 webserver.cc:460] Webserver started at http://127.0.0.1:36823/ using document root <none> and password file <none>
I20260924 00:18:14.261490 22321 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:14.261530 22321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:14.261616 22321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:14.262581 22321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "f9e361a725a04b36847e2b1a9352b0c1"
format_stamp: "Formatted at 2026-09-24 00:18:14 on dist-test-slave-cm7b"
server_key: "f9c758802de0b2b1c46e98476fd64563"
server_key_iv: "a6bb3da5a9551e81962d2a4fd455f216"
server_key_version: "encryptionkey@0"
I20260924 00:18:14.262868 22321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "f9e361a725a04b36847e2b1a9352b0c1"
format_stamp: "Formatted at 2026-09-24 00:18:14 on dist-test-slave-cm7b"
server_key: "f9c758802de0b2b1c46e98476fd64563"
server_key_iv: "a6bb3da5a9551e81962d2a4fd455f216"
server_key_version: "encryptionkey@0"
I20260924 00:18:14.264187 22321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:14.264892 22336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:14.265069 22321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:14.265152 22321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "f9e361a725a04b36847e2b1a9352b0c1"
format_stamp: "Formatted at 2026-09-24 00:18:14 on dist-test-slave-cm7b"
server_key: "f9c758802de0b2b1c46e98476fd64563"
server_key_iv: "a6bb3da5a9551e81962d2a4fd455f216"
server_key_version: "encryptionkey@0"
I20260924 00:18:14.265204 22321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:14.268308 22321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:14.270434 22321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:14.270539 22321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:14.274345 22321 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42889
I20260924 00:18:14.274410 22388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42889 every 8 connection(s)
I20260924 00:18:14.274709 22321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:14.275692 22389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:14.277817 22389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1: Bootstrap starting.
I20260924 00:18:14.278364 22389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:14.278635 22389 log.cc:824] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:14.279356 22389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1: No bootstrap required, opened a new log
I20260924 00:18:14.280857 22389 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e361a725a04b36847e2b1a9352b0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260924 00:18:14.281020 22389 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:14.281065 22389 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e361a725a04b36847e2b1a9352b0c1, State: Initialized, Role: FOLLOWER
I20260924 00:18:14.281234 22389 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e361a725a04b36847e2b1a9352b0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260924 00:18:14.281327 22389 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:14.281373 22389 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:14.281422 22389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:14.281942 22389 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e361a725a04b36847e2b1a9352b0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260924 00:18:14.282053 22389 leader_election.cc:302] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9e361a725a04b36847e2b1a9352b0c1; no voters:
I20260924 00:18:14.282222 22389 leader_election.cc:288] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:14.282253 22392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:14.282465 22389 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:14.282500 22392 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [term 1 LEADER]: Becoming Leader. State: Replica: f9e361a725a04b36847e2b1a9352b0c1, State: Running, Role: LEADER
I20260924 00:18:14.282625 22392 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e361a725a04b36847e2b1a9352b0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260924 00:18:14.282676 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22321
I20260924 00:18:14.282788 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:14.282990 1202 external_mini_cluster.cc:1428] Setting key d3ed72aa07ca989bee44b26d45fc6f49
I20260924 00:18:14.283211 22394 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9e361a725a04b36847e2b1a9352b0c1. Latest consensus state: current_term: 1 leader_uuid: "f9e361a725a04b36847e2b1a9352b0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e361a725a04b36847e2b1a9352b0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } } }
I20260924 00:18:14.283215 22393 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9e361a725a04b36847e2b1a9352b0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9e361a725a04b36847e2b1a9352b0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } } }
I20260924 00:18:14.283358 22393 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:14.283306 22394 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:14.283568 22401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:14.284068 22401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:14.285359 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:14.283492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:49183 (local address 127.0.0.1:42889)
0924 00:18:14.283725 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:14.283736 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:14.283769 (+ 33us) server_negotiation.cc:407] Connection header received
0924 00:18:14.283903 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:14.283910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:14.283941 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:14.284489 (+ 548us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:14.284927 (+ 438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:14.284930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:14.284931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:14.284935 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:14.284979 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:14.284993 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:14.285064 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:14.285141 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 00:18:14.285183 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
I20260924 00:18:14.286093 22401 catalog_manager.cc:1347] Generated new cluster ID: 856c6decf651476bafe81069e30ffe90
I20260924 00:18:14.286140 22401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:14.286281 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:14.288758 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:14.287755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:51793 (local address 127.0.0.1:42889)
0924 00:18:14.287788 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:14.287789 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:14.287915 (+ 126us) server_negotiation.cc:407] Connection header received
0924 00:18:14.287942 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:14.287943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:14.287948 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:14.288014 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:14.288552 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:14.288553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:14.288554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:14.288558 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:14.288573 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:14.288577 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:14.288632 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:14.288689 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 00:18:14.288701 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 00:18:14.302211 22401 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:14.302403 22401 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:14.310634 22401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9e361a725a04b36847e2b1a9352b0c1: Generated new TSK 0
I20260924 00:18:14.310813 22401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:14.316869 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22321
2026-09-24T00:18:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:14 dist-test-slave-cm7b krb5kdc[22420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:14 dist-test-slave-cm7b krb5kdc[22420](info): set up 2 sockets
Sep 24 00:18:14 dist-test-slave-cm7b krb5kdc[22420](info): commencing operation
krb5kdc: starting...
W20260924 00:18:15.343370 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:15 dist-test-slave-cm7b krb5kdc[22420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:15.455883 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:15.533051 22433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:15.533264 22433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:15.533293 22433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:15.534428 22433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:15.534469 22433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:15.534484 22433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:15.534538 22433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:15.534562 22433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:15.536060 22433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:15.536309 22433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:15.536530 22433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:15.541180 22441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:15.541249 22439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:15.541218 22438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:15.541355 22433 server_base.cc:1034] running on GCE node
I20260924 00:18:15.541680 22433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:15.541977 22433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:15.543149 22433 hybrid_clock.cc:648] HybridClock initialized: now 1790209095543136 us; error 40 us; skew 500 ppm
Sep 24 00:18:15 dist-test-slave-cm7b krb5kdc[22420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209095, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:15.551519 22433 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:15.551978 22433 webserver.cc:460] Webserver started at http://127.0.0.1:42781/ using document root <none> and password file <none>
I20260924 00:18:15.552196 22433 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:15.552261 22433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:15.552413 22433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:15.553427 22433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "45a468e95bc44aecb58302761495744b"
format_stamp: "Formatted at 2026-09-24 00:18:15 on dist-test-slave-cm7b"
server_key: "e958b36bc116aa98f29c757bf0989c02"
server_key_iv: "a5fad0d93593d4ad02dcaa4320a09fe3"
server_key_version: "encryptionkey@0"
I20260924 00:18:15.553781 22433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "45a468e95bc44aecb58302761495744b"
format_stamp: "Formatted at 2026-09-24 00:18:15 on dist-test-slave-cm7b"
server_key: "e958b36bc116aa98f29c757bf0989c02"
server_key_iv: "a5fad0d93593d4ad02dcaa4320a09fe3"
server_key_version: "encryptionkey@0"
I20260924 00:18:15.555049 22433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 00:18:15.555848 22448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:15.556018 22433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:15.556113 22433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "45a468e95bc44aecb58302761495744b"
format_stamp: "Formatted at 2026-09-24 00:18:15 on dist-test-slave-cm7b"
server_key: "e958b36bc116aa98f29c757bf0989c02"
server_key_iv: "a5fad0d93593d4ad02dcaa4320a09fe3"
server_key_version: "encryptionkey@0"
I20260924 00:18:15.556196 22433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:15.564884 22433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:15.566524 22433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:15.566673 22433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:15.570724 22433 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45831
I20260924 00:18:15.570755 22500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45831 every 8 connection(s)
I20260924 00:18:15.571065 22433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:15.572036 22501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:15.574234 22501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b: Bootstrap starting.
I20260924 00:18:15.574797 22501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:15.575073 22501 log.cc:824] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:15.575788 22501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b: No bootstrap required, opened a new log
I20260924 00:18:15.577347 22501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a468e95bc44aecb58302761495744b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45831 } }
I20260924 00:18:15.577476 22501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:15.577497 22501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a468e95bc44aecb58302761495744b, State: Initialized, Role: FOLLOWER
I20260924 00:18:15.577666 22501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a468e95bc44aecb58302761495744b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45831 } }
I20260924 00:18:15.577734 22501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:15.577776 22501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:15.577823 22501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:15.578356 22501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a468e95bc44aecb58302761495744b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45831 } }
I20260924 00:18:15.578462 22501 leader_election.cc:302] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45a468e95bc44aecb58302761495744b; no voters:
I20260924 00:18:15.578640 22501 leader_election.cc:288] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:15.578667 22504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:15.578847 22504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [term 1 LEADER]: Becoming Leader. State: Replica: 45a468e95bc44aecb58302761495744b, State: Running, Role: LEADER
I20260924 00:18:15.578891 22501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:15.578961 22504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a468e95bc44aecb58302761495744b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45831 } }
I20260924 00:18:15.579407 22506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45a468e95bc44aecb58302761495744b. Latest consensus state: current_term: 1 leader_uuid: "45a468e95bc44aecb58302761495744b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a468e95bc44aecb58302761495744b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45831 } } }
I20260924 00:18:15.579483 22506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:15.579543 22505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45a468e95bc44aecb58302761495744b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a468e95bc44aecb58302761495744b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45831 } } }
I20260924 00:18:15.579624 22505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:15.580006 22512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:15.580276 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22433
I20260924 00:18:15.580404 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:15.580530 22512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:15.580597 1202 external_mini_cluster.cc:1428] Setting key c3729941eb3c80b2d8b65f51dab2b628
I20260924 00:18:15.581992 22512 catalog_manager.cc:1347] Generated new cluster ID: 4f5461502de24d96907bdfd1016f5739
I20260924 00:18:15.582042 22512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:18:15 dist-test-slave-cm7b krb5kdc[22420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 00:18:15.595115 22512 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:15.595655 22512 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:15.597126 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:15.581178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:47641 (local address 127.0.0.1:45831)
0924 00:18:15.581682 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:15.581691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:15.581721 (+ 30us) server_negotiation.cc:407] Connection header received
0924 00:18:15.581841 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:15.581848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:15.581902 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:15.582039 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:15.582893 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:15.584278 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:15.585360 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:15.585923 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:15.586020 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:15.593303 (+ 7283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:15.593307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:15.593307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:15.593318 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:15.595480 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:15.595932 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:15.595935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:15.595935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:15.596048 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:15.596269 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:15.596270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:15.596270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:15.596545 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:15.596683 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:15.596808 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 00:18:15.596904 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":70,"threads_started":1}
I20260924 00:18:15.598062 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:15.601733 22512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45a468e95bc44aecb58302761495744b: Generated new TSK 0
I20260924 00:18:15.601907 22512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:15.606014 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:15.599554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:45009 (local address 127.0.0.1:45831)
0924 00:18:15.599586 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:15.599588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:15.599599 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:18:15.599677 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:15.599677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:15.599686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:15.599735 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:15.600316 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:15.601056 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:15.602116 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:15.602324 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:15.602376 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:15.603666 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:15.603667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:15.603667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:15.603672 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:15.604955 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:15.605352 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:15.605353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:15.605353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:15.605399 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:15.605611 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:15.605612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:15.605612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:15.605696 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:15.605765 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:15.605920 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 00:18:15.605963 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 00:18:15.607864 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22433
W20260924 00:18:15.611707 22418 connection.cc:535] client connection to 127.0.0.1:45831 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45831): Connection reset by peer (error 104)
2026-09-24T00:18:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:15 dist-test-slave-cm7b krb5kdc[22532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:15 dist-test-slave-cm7b krb5kdc[22532](info): set up 2 sockets
Sep 24 00:18:15 dist-test-slave-cm7b krb5kdc[22532](info): commencing operation
krb5kdc: starting...
W20260924 00:18:16.647445 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:16 dist-test-slave-cm7b krb5kdc[22532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:16.759263 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:16.837756 22545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:16.837946 22545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:16.837970 22545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:16.839088 22545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:16.839130 22545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:16.839144 22545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:16.839156 22545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:16.839223 22545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:16.840653 22545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39843
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:16.840888 22545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:16.841099 22545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:16.845714 22550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:16.845853 22551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:16.845728 22553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:16.845726 22545 server_base.cc:1034] running on GCE node
I20260924 00:18:16.846132 22545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:16.846397 22545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:16.847533 22545 hybrid_clock.cc:648] HybridClock initialized: now 1790209096847521 us; error 28 us; skew 500 ppm
Sep 24 00:18:16 dist-test-slave-cm7b krb5kdc[22532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209096, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:16.849809 22545 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:16.850208 22545 webserver.cc:460] Webserver started at http://127.0.0.1:43715/ using document root <none> and password file <none>
I20260924 00:18:16.850442 22545 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:16.850512 22545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:16.850662 22545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:16.851717 22545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "290fc1ff51384c85989f411004c0be06"
format_stamp: "Formatted at 2026-09-24 00:18:16 on dist-test-slave-cm7b"
server_key: "ffe31fd2efb370586e9e758430d7d6e1"
server_key_iv: "3434b5884f1ff81bb0604fafc4374e26"
server_key_version: "encryptionkey@0"
I20260924 00:18:16.852070 22545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "290fc1ff51384c85989f411004c0be06"
format_stamp: "Formatted at 2026-09-24 00:18:16 on dist-test-slave-cm7b"
server_key: "ffe31fd2efb370586e9e758430d7d6e1"
server_key_iv: "3434b5884f1ff81bb0604fafc4374e26"
server_key_version: "encryptionkey@0"
I20260924 00:18:16.853322 22545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:16.854017 22560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:16.854182 22545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:16.854276 22545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "290fc1ff51384c85989f411004c0be06"
format_stamp: "Formatted at 2026-09-24 00:18:16 on dist-test-slave-cm7b"
server_key: "ffe31fd2efb370586e9e758430d7d6e1"
server_key_iv: "3434b5884f1ff81bb0604fafc4374e26"
server_key_version: "encryptionkey@0"
I20260924 00:18:16.854353 22545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:16.864894 22545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:16.866724 22545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:16.866874 22545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:16.870836 22545 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39843
I20260924 00:18:16.870867 22612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39843 every 8 connection(s)
I20260924 00:18:16.871241 22545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:16.872099 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:16.873082 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22545
I20260924 00:18:16.873226 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:16.873409 1202 external_mini_cluster.cc:1428] Setting key d5c935f8c5995a7244b45fae1afdfccb
I20260924 00:18:16.874601 22613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06: Bootstrap starting.
I20260924 00:18:16.875165 22613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:16.875492 22613 log.cc:824] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:16.876180 22613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06: No bootstrap required, opened a new log
I20260924 00:18:16.877897 22613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290fc1ff51384c85989f411004c0be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39843 } }
I20260924 00:18:16.878046 22613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:16.878090 22613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 290fc1ff51384c85989f411004c0be06, State: Initialized, Role: FOLLOWER
I20260924 00:18:16.878271 22613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290fc1ff51384c85989f411004c0be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39843 } }
I20260924 00:18:16.878368 22613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:16.878428 22613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:16.878482 22613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:16.878705 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:16.873934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:46991 (local address 127.0.0.1:39843)
0924 00:18:16.874108 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:16.874121 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:16.874144 (+ 23us) server_negotiation.cc:407] Connection header received
0924 00:18:16.874529 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:16.874535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:16.874579 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:16.874726 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:16.874921 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:16.876030 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:16.876873 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:16.877289 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:16.877415 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:16.878064 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:16.878068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:16.878068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:16.878286 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:16.878314 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:16.878324 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:16.878420 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:16.878496 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 00:18:16.878524 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260924 00:18:16.879211 22613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290fc1ff51384c85989f411004c0be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39843 } }
I20260924 00:18:16.879333 22613 leader_election.cc:302] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 290fc1ff51384c85989f411004c0be06; no voters:
I20260924 00:18:16.879496 22613 leader_election.cc:288] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:16.879527 22618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:16.879701 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:16.879711 22618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [term 1 LEADER]: Becoming Leader. State: Replica: 290fc1ff51384c85989f411004c0be06, State: Running, Role: LEADER
I20260924 00:18:16.879739 22613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:16.879815 22618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290fc1ff51384c85989f411004c0be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39843 } }
I20260924 00:18:16.880100 22620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 290fc1ff51384c85989f411004c0be06. Latest consensus state: current_term: 1 leader_uuid: "290fc1ff51384c85989f411004c0be06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290fc1ff51384c85989f411004c0be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39843 } } }
I20260924 00:18:16.880164 22620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:16.880733 22625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:16.881201 22625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:16.882638 22625 catalog_manager.cc:1347] Generated new cluster ID: 3af9f14638c64f0cbfd2a281c1f87cd7
I20260924 00:18:16.882688 22625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:16.887252 22619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "290fc1ff51384c85989f411004c0be06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290fc1ff51384c85989f411004c0be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39843 } } }
I20260924 00:18:16.887409 22619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:16.890659 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:16.883513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:57811 (local address 127.0.0.1:39843)
0924 00:18:16.887263 (+ 3750us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:16.887265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:16.887370 (+ 105us) server_negotiation.cc:407] Connection header received
0924 00:18:16.887453 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:16.887454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:16.887464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:16.887490 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:16.887669 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:16.888386 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:16.889347 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:16.889642 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:16.889675 (+ 33us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:16.890429 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:16.890430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:16.890431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:16.890434 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:16.890446 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:16.890450 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:16.890477 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:16.890584 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 00:18:16.890593 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3731}
I20260924 00:18:16.901364 22625 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:16.901849 22625 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:16.906741 22625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 290fc1ff51384c85989f411004c0be06: Generated new TSK 0
I20260924 00:18:16.906910 22625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:16.913650 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22545
W20260924 00:18:16.917451 22530 connection.cc:535] client connection to 127.0.0.1:39843 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39843): Connection reset by peer (error 104)
2026-09-24T00:18:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:16 dist-test-slave-cm7b krb5kdc[22644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:16 dist-test-slave-cm7b krb5kdc[22644](info): set up 2 sockets
Sep 24 00:18:16 dist-test-slave-cm7b krb5kdc[22644](info): commencing operation
krb5kdc: starting...
W20260924 00:18:17.947453 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:18 dist-test-slave-cm7b krb5kdc[22644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 00:18:18.058270 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:18.136507 22657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:18.136747 22657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:18.136787 22657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:18.137974 22657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:18.138047 22657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:18.138072 22657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:18.138100 22657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:18.138128 22657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:18.139621 22657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43529
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:18.139895 22657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:18.140113 22657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:18.144872 22662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:18.145027 22657 server_base.cc:1034] running on GCE node
W20260924 00:18:18.144868 22663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:18.145032 22665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:18.145404 22657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:18.145684 22657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:18.146822 22657 hybrid_clock.cc:648] HybridClock initialized: now 1790209098146807 us; error 35 us; skew 500 ppm
Sep 24 00:18:18 dist-test-slave-cm7b krb5kdc[22644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:18.149164 22657 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 00:18:18.149577 22657 webserver.cc:460] Webserver started at http://127.0.0.1:42489/ using document root <none> and password file <none>
I20260924 00:18:18.149796 22657 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:18.149861 22657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:18.150017 22657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:18.151031 22657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "f7b89e6dda034eb3b61cb8448462a8d9"
format_stamp: "Formatted at 2026-09-24 00:18:18 on dist-test-slave-cm7b"
server_key: "233b50d3bc5b10e8cdb94f2e44e93e95"
server_key_iv: "de762dca0b038f426b433f22cac683e8"
server_key_version: "encryptionkey@0"
I20260924 00:18:18.151408 22657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "f7b89e6dda034eb3b61cb8448462a8d9"
format_stamp: "Formatted at 2026-09-24 00:18:18 on dist-test-slave-cm7b"
server_key: "233b50d3bc5b10e8cdb94f2e44e93e95"
server_key_iv: "de762dca0b038f426b433f22cac683e8"
server_key_version: "encryptionkey@0"
I20260924 00:18:18.152655 22657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:18:18.153349 22672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:18.153512 22657 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:18.153607 22657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "f7b89e6dda034eb3b61cb8448462a8d9"
format_stamp: "Formatted at 2026-09-24 00:18:18 on dist-test-slave-cm7b"
server_key: "233b50d3bc5b10e8cdb94f2e44e93e95"
server_key_iv: "de762dca0b038f426b433f22cac683e8"
server_key_version: "encryptionkey@0"
I20260924 00:18:18.153684 22657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:18.156876 22657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:18.158716 22657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:18.158864 22657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:18.162676 22657 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43529
I20260924 00:18:18.162724 22724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43529 every 8 connection(s)
I20260924 00:18:18.163049 22657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:18.164005 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:18.166090 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9: Bootstrap starting.
I20260924 00:18:18.166630 22725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:18.166909 22725 log.cc:824] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:18.167600 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9: No bootstrap required, opened a new log
I20260924 00:18:18.169098 22725 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43529 } }
I20260924 00:18:18.169232 22725 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:18.169256 22725 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b89e6dda034eb3b61cb8448462a8d9, State: Initialized, Role: FOLLOWER
I20260924 00:18:18.169399 22725 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43529 } }
I20260924 00:18:18.169482 22725 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:18.169523 22725 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:18.169567 22725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:18.170065 22725 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43529 } }
I20260924 00:18:18.170173 22725 leader_election.cc:302] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7b89e6dda034eb3b61cb8448462a8d9; no voters:
I20260924 00:18:18.170351 22725 leader_election.cc:288] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:18.170425 22728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:18.170580 22725 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:18.170568 22728 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [term 1 LEADER]: Becoming Leader. State: Replica: f7b89e6dda034eb3b61cb8448462a8d9, State: Running, Role: LEADER
I20260924 00:18:18.170745 22728 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43529 } }
I20260924 00:18:18.171334 22730 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7b89e6dda034eb3b61cb8448462a8d9. Latest consensus state: current_term: 1 leader_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43529 } } }
I20260924 00:18:18.171432 22730 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:18.171643 22729 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b89e6dda034eb3b61cb8448462a8d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43529 } } }
I20260924 00:18:18.171734 22729 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:18.171977 22737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:18.172422 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22657
I20260924 00:18:18.172465 22737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:18.172545 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:18.172736 1202 external_mini_cluster.cc:1428] Setting key 09117af996713ac2e79365046ec314bf
I20260924 00:18:18.173981 22737 catalog_manager.cc:1347] Generated new cluster ID: cdcbac14fcf741d5a5ae4a98a5afd522
I20260924 00:18:18.174036 22737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:18.176839 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:18.173133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:50757 (local address 127.0.0.1:43529)
0924 00:18:18.175345 (+ 2212us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:18.175356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:18.175394 (+ 38us) server_negotiation.cc:407] Connection header received
0924 00:18:18.175563 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:18.175568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:18.175590 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:18.175714 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:18.176425 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:18.176429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:18.176429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:18.176433 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:18.176476 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:18.176487 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:18.176553 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:18.176625 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 00:18:18.176670 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2161,"thread_start_us":63,"threads_started":1}
I20260924 00:18:18.177673 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:18.180238 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:18.178923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.40:35523 (local address 127.0.0.1:43529)
0924 00:18:18.178959 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:18.178961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:18.179353 (+ 392us) server_negotiation.cc:407] Connection header received
0924 00:18:18.179426 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:18.179427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:18.179432 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:18.179480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:18.180017 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:18.180018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:18.180018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:18.180023 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:18.180035 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:18.180039 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:18.180088 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:18.180139 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 00:18:18.180154 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:18.187345 22737 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:18.187489 22737 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:18.193140 22737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7b89e6dda034eb3b61cb8448462a8d9: Generated new TSK 0
I20260924 00:18:18.193300 22737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:18.198103 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22657
2026-09-24T00:18:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:18 dist-test-slave-cm7b krb5kdc[22756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:18 dist-test-slave-cm7b krb5kdc[22756](info): set up 2 sockets
Sep 24 00:18:18 dist-test-slave-cm7b krb5kdc[22756](info): commencing operation
krb5kdc: starting...
W20260924 00:18:19.239424 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:19 dist-test-slave-cm7b krb5kdc[22756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:19.354456 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44749
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:44749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:19.433139 22769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:19.433382 22769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:19.433423 22769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:19.434540 22769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:19.434607 22769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:19.434633 22769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:19.434660 22769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:19.434693 22769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:19.436137 22769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:44749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:44749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:19.436400 22769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:19.436625 22769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:19.441431 22774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:19.441411 22777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:19.441411 22775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:19.441680 22769 server_base.cc:1034] running on GCE node
I20260924 00:18:19.441926 22769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:19.442194 22769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:19.443327 22769 hybrid_clock.cc:648] HybridClock initialized: now 1790209099443317 us; error 28 us; skew 500 ppm
Sep 24 00:18:19 dist-test-slave-cm7b krb5kdc[22756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209099, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:19.445583 22769 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:19.446030 22769 webserver.cc:460] Webserver started at http://127.1.44.190:44771/ using document root <none> and password file <none>
I20260924 00:18:19.446260 22769 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:19.446327 22769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:19.446476 22769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:19.447521 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "8874f46694ca47f584e6351af6df4c21"
format_stamp: "Formatted at 2026-09-24 00:18:19 on dist-test-slave-cm7b"
server_key: "a9b5fe033fde1ea75d5c778396237cb8"
server_key_iv: "fcfbd8d50bb022b7b272d3218199d14a"
server_key_version: "encryptionkey@0"
I20260924 00:18:19.447862 22769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "8874f46694ca47f584e6351af6df4c21"
format_stamp: "Formatted at 2026-09-24 00:18:19 on dist-test-slave-cm7b"
server_key: "a9b5fe033fde1ea75d5c778396237cb8"
server_key_iv: "fcfbd8d50bb022b7b272d3218199d14a"
server_key_version: "encryptionkey@0"
I20260924 00:18:19.449101 22769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:19.449751 22784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:19.449920 22769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:19.450016 22769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "8874f46694ca47f584e6351af6df4c21"
format_stamp: "Formatted at 2026-09-24 00:18:19 on dist-test-slave-cm7b"
server_key: "a9b5fe033fde1ea75d5c778396237cb8"
server_key_iv: "fcfbd8d50bb022b7b272d3218199d14a"
server_key_version: "encryptionkey@0"
I20260924 00:18:19.450094 22769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:19.460994 22769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:19.462636 22769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:19.462793 22769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:19.466737 22769 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:44749
I20260924 00:18:19.466766 22836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:44749 every 8 connection(s)
I20260924 00:18:19.467088 22769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:19.467963 22837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:19.468417 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22769
I20260924 00:18:19.468547 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:19.468730 1202 external_mini_cluster.cc:1428] Setting key 839fd42915f4348d77765da9bc095692
I20260924 00:18:19.470841 22837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21: Bootstrap starting.
I20260924 00:18:19.471515 22837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:19.471815 22837 log.cc:824] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:19.472478 22837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21: No bootstrap required, opened a new log
I20260924 00:18:19.474722 22837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874f46694ca47f584e6351af6df4c21" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44749 } }
I20260924 00:18:19.474848 22837 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:19.474870 22837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8874f46694ca47f584e6351af6df4c21, State: Initialized, Role: FOLLOWER
I20260924 00:18:19.475059 22837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874f46694ca47f584e6351af6df4c21" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44749 } }
I20260924 00:18:19.475144 22837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:19.475173 22837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:19.475256 22837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 0 FOLLOWER]: Advancing to term 1
Sep 24 00:18:19 dist-test-slave-cm7b krb5kdc[22756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:19.475930 22837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874f46694ca47f584e6351af6df4c21" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44749 } }
I20260924 00:18:19.476046 22837 leader_election.cc:302] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8874f46694ca47f584e6351af6df4c21; no voters:
I20260924 00:18:19.476210 22837 leader_election.cc:288] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:19.476264 22842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:19.476460 22842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [term 1 LEADER]: Becoming Leader. State: Replica: 8874f46694ca47f584e6351af6df4c21, State: Running, Role: LEADER
I20260924 00:18:19.476473 22837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:19.476559 22842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874f46694ca47f584e6351af6df4c21" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44749 } }
I20260924 00:18:19.476899 22843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8874f46694ca47f584e6351af6df4c21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874f46694ca47f584e6351af6df4c21" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44749 } } }
I20260924 00:18:19.476994 22843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:19.476890 22842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8874f46694ca47f584e6351af6df4c21. Latest consensus state: current_term: 1 leader_uuid: "8874f46694ca47f584e6351af6df4c21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874f46694ca47f584e6351af6df4c21" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 44749 } } }
I20260924 00:18:19.477275 22842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:19.477182 22849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:19.478538 22849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:19.480175 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:19.469238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45755 (local address 127.1.44.190:44749)
0924 00:18:19.469416 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:19.469434 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:19.469469 (+ 35us) server_negotiation.cc:407] Connection header received
0924 00:18:19.469982 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:19.469990 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:19.470049 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:19.470194 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:19.470749 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:19.472122 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:19.472901 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:19.473569 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:19.473681 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:19.476181 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:19.476189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:19.476190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:19.476214 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:19.478297 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:19.478706 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:19.478709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:19.478710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:19.478793 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:19.479028 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:19.479029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:19.479029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:19.479352 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:19.479863 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:19.479933 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 00:18:19.480005 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260924 00:18:19.480203 22849 catalog_manager.cc:1347] Generated new cluster ID: 74cbe0dd4e4b40ceaf56ce82573a995a
I20260924 00:18:19.480247 22849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:19.481117 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:19.489105 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:19.482410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50839 (local address 127.1.44.190:44749)
0924 00:18:19.482517 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:19.482519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:19.483383 (+ 864us) server_negotiation.cc:407] Connection header received
0924 00:18:19.483455 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:19.483456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:19.483464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:19.483507 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:19.483997 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:19.484455 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:19.485141 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:19.485370 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:19.485443 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:19.486557 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:19.486558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:19.486558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:19.486566 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:19.487970 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:19.488336 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:19.488337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:19.488337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:19.488389 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:19.488612 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:19.488613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:19.488614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:19.488719 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:19.488839 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:19.488949 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 00:18:19.489008 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260924 00:18:19.491287 22849 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:19.491765 22849 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:19.495644 22849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8874f46694ca47f584e6351af6df4c21: Generated new TSK 0
I20260924 00:18:19.495821 22849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:19.501688 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22769
W20260924 00:18:19.505805 22754 connection.cc:535] client connection to 127.1.44.190:44749 recv error: Network error: failed to read from TLS socket (remote: 127.1.44.190:44749): Connection reset by peer (error 104)
2026-09-24T00:18:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:19 dist-test-slave-cm7b krb5kdc[22868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:19 dist-test-slave-cm7b krb5kdc[22868](info): set up 2 sockets
Sep 24 00:18:19 dist-test-slave-cm7b krb5kdc[22868](info): commencing operation
krb5kdc: starting...
W20260924 00:18:20.523100 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:20 dist-test-slave-cm7b krb5kdc[22868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:20.634423 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36827
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:45533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:36827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:20.712723 22881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:20.712906 22881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:20.712929 22881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:20.714043 22881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:20.714084 22881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:20.714098 22881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:20.714111 22881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:20.714145 22881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:20.715569 22881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:45533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:36827
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:36827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:20.715793 22881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:20.715997 22881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:20.720490 22889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:20.720503 22887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:20.720644 22881 server_base.cc:1034] running on GCE node
W20260924 00:18:20.720503 22886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:20.720937 22881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:20.721222 22881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:20.722414 22881 hybrid_clock.cc:648] HybridClock initialized: now 1790209100722397 us; error 27 us; skew 500 ppm
Sep 24 00:18:20 dist-test-slave-cm7b krb5kdc[22868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209100, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:20.730542 22881 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:20.730909 22881 webserver.cc:460] Webserver started at http://127.1.44.190:40701/ using document root <none> and password file <none>
I20260924 00:18:20.731078 22881 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:20.731117 22881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:20.731264 22881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:20.732239 22881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "2e56fb1ad985457c817ee1b7fa506508"
format_stamp: "Formatted at 2026-09-24 00:18:20 on dist-test-slave-cm7b"
server_key: "7cbdbd6031adcaa2ad754e26b853ac83"
server_key_iv: "5de7abf03a702ad8a78186c31ded471b"
server_key_version: "encryptionkey@0"
I20260924 00:18:20.732529 22881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "2e56fb1ad985457c817ee1b7fa506508"
format_stamp: "Formatted at 2026-09-24 00:18:20 on dist-test-slave-cm7b"
server_key: "7cbdbd6031adcaa2ad754e26b853ac83"
server_key_iv: "5de7abf03a702ad8a78186c31ded471b"
server_key_version: "encryptionkey@0"
I20260924 00:18:20.733726 22881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 00:18:20.734370 22896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:20.734521 22881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:20.734604 22881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "2e56fb1ad985457c817ee1b7fa506508"
format_stamp: "Formatted at 2026-09-24 00:18:20 on dist-test-slave-cm7b"
server_key: "7cbdbd6031adcaa2ad754e26b853ac83"
server_key_iv: "5de7abf03a702ad8a78186c31ded471b"
server_key_version: "encryptionkey@0"
I20260924 00:18:20.734656 22881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:20.746361 22881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:20.748212 22881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:20.748314 22881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:20.752076 22881 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:36827
I20260924 00:18:20.752125 22948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:36827 every 8 connection(s)
I20260924 00:18:20.752422 22881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:20.753312 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:20.755517 22949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508: Bootstrap starting.
I20260924 00:18:20.756059 22949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:20.756346 22949 log.cc:824] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:20.757001 22949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508: No bootstrap required, opened a new log
I20260924 00:18:20.758383 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22881
I20260924 00:18:20.758482 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:20.758705 1202 external_mini_cluster.cc:1428] Setting key 5697974a1b87e088875f640c927986a9
I20260924 00:18:20.758673 22949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56fb1ad985457c817ee1b7fa506508" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36827 } }
I20260924 00:18:20.758858 22949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:20.758901 22949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e56fb1ad985457c817ee1b7fa506508, State: Initialized, Role: FOLLOWER
I20260924 00:18:20.759043 22949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56fb1ad985457c817ee1b7fa506508" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36827 } }
I20260924 00:18:20.759136 22949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:20.759169 22949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:20.759244 22949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:20.760123 22949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56fb1ad985457c817ee1b7fa506508" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36827 } }
I20260924 00:18:20.760262 22949 leader_election.cc:302] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e56fb1ad985457c817ee1b7fa506508; no voters:
I20260924 00:18:20.760447 22949 leader_election.cc:288] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:20.760492 22954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:20.760644 22954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [term 1 LEADER]: Becoming Leader. State: Replica: 2e56fb1ad985457c817ee1b7fa506508, State: Running, Role: LEADER
I20260924 00:18:20.760703 22949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:20.760787 22954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56fb1ad985457c817ee1b7fa506508" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36827 } }
I20260924 00:18:20.761202 22955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e56fb1ad985457c817ee1b7fa506508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56fb1ad985457c817ee1b7fa506508" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36827 } } }
I20260924 00:18:20.761298 22955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:20.761447 22956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e56fb1ad985457c817ee1b7fa506508. Latest consensus state: current_term: 1 leader_uuid: "2e56fb1ad985457c817ee1b7fa506508" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56fb1ad985457c817ee1b7fa506508" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 36827 } } }
I20260924 00:18:20.761509 22956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:20.761739 22962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:20.762209 22962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:20.763713 22962 catalog_manager.cc:1347] Generated new cluster ID: ffa6bd6ae4db4b9e953fc7e1e6f8a96f
I20260924 00:18:20.763761 22962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:20.764621 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:20.759174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60613 (local address 127.1.44.190:36827)
0924 00:18:20.759578 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:20.759586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:20.759608 (+ 22us) server_negotiation.cc:407] Connection header received
0924 00:18:20.759717 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:20.759722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:20.759762 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:20.759862 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:20.760171 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:20.761411 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:20.762543 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:20.763120 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:20.763252 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:20.763942 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:20.763946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:20.763947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:20.764151 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:20.764184 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:20.764197 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:20.764273 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:20.764412 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 00:18:20.764437 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":118,"threads_started":1}
I20260924 00:18:20.765574 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:20.770574 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:20.767016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59171 (local address 127.1.44.190:36827)
0924 00:18:20.767069 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:20.767070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:20.767138 (+ 68us) server_negotiation.cc:407] Connection header received
0924 00:18:20.767269 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:20.767270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:20.767280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:20.767331 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:20.767474 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:20.768162 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:20.769270 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:20.769589 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:20.769651 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:20.770325 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:20.770326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:20.770327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:20.770331 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:20.770343 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:20.770347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:20.770406 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:20.770496 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 00:18:20.770507 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 00:18:20.785235 22962 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:20.785964 22962 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:20.791401 22962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e56fb1ad985457c817ee1b7fa506508: Generated new TSK 0
I20260924 00:18:20.791580 22962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:20.799134 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22881
W20260924 00:18:20.803025 22866 connection.cc:535] client connection to 127.1.44.190:36827 recv error: Network error: failed to read from TLS socket (remote: 127.1.44.190:36827): Connection reset by peer (error 104)
2026-09-24T00:18:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:20 dist-test-slave-cm7b krb5kdc[22980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:20 dist-test-slave-cm7b krb5kdc[22980](info): set up 2 sockets
Sep 24 00:18:20 dist-test-slave-cm7b krb5kdc[22980](info): commencing operation
krb5kdc: starting...
W20260924 00:18:21.831173 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:21 dist-test-slave-cm7b krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:21.942080 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:45997
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:45997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:22.019582 22993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:22.019796 22993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:22.019832 22993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:22.020956 22993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:22.021023 22993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:22.021047 22993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:22.021076 22993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:22.021107 22993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:22.022526 22993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:45997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:22.022786 22993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:22.023006 22993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:22.027580 23001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:22.027593 22998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:22.027599 22999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:22.027738 22993 server_base.cc:1034] running on GCE node
I20260924 00:18:22.027966 22993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:22.028188 22993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:22.029359 22993 hybrid_clock.cc:648] HybridClock initialized: now 1790209102029343 us; error 26 us; skew 500 ppm
Sep 24 00:18:22 dist-test-slave-cm7b krb5kdc[22980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209102, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:22.031981 22993 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:22.032447 22993 webserver.cc:460] Webserver started at http://127.1.44.190:45375/ using document root <none> and password file <none>
I20260924 00:18:22.032657 22993 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:22.032723 22993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:22.032868 22993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:22.033867 22993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "09dcccb8a8b84731bfbb39b3b8011c54"
format_stamp: "Formatted at 2026-09-24 00:18:22 on dist-test-slave-cm7b"
server_key: "c433c674a06f90e90f741fd8a260bd90"
server_key_iv: "859679a2ee51c0ed5f6176db84d997bb"
server_key_version: "encryptionkey@0"
I20260924 00:18:22.034217 22993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "09dcccb8a8b84731bfbb39b3b8011c54"
format_stamp: "Formatted at 2026-09-24 00:18:22 on dist-test-slave-cm7b"
server_key: "c433c674a06f90e90f741fd8a260bd90"
server_key_iv: "859679a2ee51c0ed5f6176db84d997bb"
server_key_version: "encryptionkey@0"
I20260924 00:18:22.035522 22993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:22.036216 23008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:22.036383 22993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:22.036489 22993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "09dcccb8a8b84731bfbb39b3b8011c54"
format_stamp: "Formatted at 2026-09-24 00:18:22 on dist-test-slave-cm7b"
server_key: "c433c674a06f90e90f741fd8a260bd90"
server_key_iv: "859679a2ee51c0ed5f6176db84d997bb"
server_key_version: "encryptionkey@0"
I20260924 00:18:22.036558 22993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:22.039786 22993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:22.041970 22993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:22.042120 22993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:22.045981 22993 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:45997
I20260924 00:18:22.046032 23060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:45997 every 8 connection(s)
I20260924 00:18:22.046341 22993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:22.047380 23061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:22.049961 23061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54: Bootstrap starting.
I20260924 00:18:22.050518 23061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:22.050794 23061 log.cc:824] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:22.051478 23061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54: No bootstrap required, opened a new log
I20260924 00:18:22.052965 23061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45997 } }
I20260924 00:18:22.053102 23061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:22.053124 23061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dcccb8a8b84731bfbb39b3b8011c54, State: Initialized, Role: FOLLOWER
I20260924 00:18:22.053323 23061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45997 } }
I20260924 00:18:22.053385 23061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:22.053439 23061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:22.053490 23061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:22.054279 23061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45997 } }
I20260924 00:18:22.054390 23061 leader_election.cc:302] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09dcccb8a8b84731bfbb39b3b8011c54; no voters:
I20260924 00:18:22.054566 23061 leader_election.cc:288] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:22.054625 23064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:22.054791 23064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [term 1 LEADER]: Becoming Leader. State: Replica: 09dcccb8a8b84731bfbb39b3b8011c54, State: Running, Role: LEADER
I20260924 00:18:22.054824 23061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:22.054905 23064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45997 } }
I20260924 00:18:22.055280 23066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09dcccb8a8b84731bfbb39b3b8011c54. Latest consensus state: current_term: 1 leader_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45997 } } }
I20260924 00:18:22.055364 23066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:22.055537 23065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dcccb8a8b84731bfbb39b3b8011c54" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45997 } } }
I20260924 00:18:22.055652 23065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:22.055575 23073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:22.056145 23073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:22.056151 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 22993
I20260924 00:18:22.056272 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:22.056458 1202 external_mini_cluster.cc:1428] Setting key ee19ec5e8a45bac3255e35f2884a97ba
I20260924 00:18:22.058192 23073 catalog_manager.cc:1347] Generated new cluster ID: 301285d9a46f4dbc8e15d2322d25dabe
I20260924 00:18:22.058246 23073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:22.058607 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:22.057073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34735 (local address 127.1.44.190:45997)
0924 00:18:22.057232 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:22.057241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:22.057275 (+ 34us) server_negotiation.cc:407] Connection header received
0924 00:18:22.057377 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:22.057384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:22.057426 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:22.057558 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:22.058192 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:22.058196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:22.058196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:22.058200 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:22.058242 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:22.058259 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:22.058319 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:22.058421 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 00:18:22.058462 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260924 00:18:22.059657 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:22.068297 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:22.063424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45949 (local address 127.1.44.190:45997)
0924 00:18:22.063455 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:22.063458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:22.067372 (+ 3914us) server_negotiation.cc:407] Connection header received
0924 00:18:22.067442 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:22.067443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:22.067449 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:22.067493 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:22.068074 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:22.068075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:22.068075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:22.068079 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:22.068093 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:22.068097 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:22.068143 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:22.068204 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 00:18:22.068218 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 00:18:22.071262 23073 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:22.071415 23073 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:22.079020 23073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09dcccb8a8b84731bfbb39b3b8011c54: Generated new TSK 0
I20260924 00:18:22.079171 23073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:22.084004 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 22993
2026-09-24T00:18:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:22 dist-test-slave-cm7b krb5kdc[23092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:22 dist-test-slave-cm7b krb5kdc[23092](info): set up 2 sockets
Sep 24 00:18:22 dist-test-slave-cm7b krb5kdc[23092](info): commencing operation
krb5kdc: starting...
W20260924 00:18:23.115350 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:23 dist-test-slave-cm7b krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:23.226816 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:38667
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:33011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:38667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:23.305658 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:23.305858 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:23.305881 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:23.306937 23105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:23.306972 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:23.306985 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:23.306998 23105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:23.307010 23105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:23.308641 23105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:33011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:38667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:38667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:23.308888 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:23.309219 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:23.315029 23111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:23.315037 23110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:23.315112 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:23.315122 23105 server_base.cc:1034] running on GCE node
I20260924 00:18:23.315618 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:23.316000 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:23.317164 23105 hybrid_clock.cc:648] HybridClock initialized: now 1790209103317131 us; error 44 us; skew 500 ppm
Sep 24 00:18:23 dist-test-slave-cm7b krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209103, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:23.319798 23105 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:23.320279 23105 webserver.cc:460] Webserver started at http://127.1.44.190:38841/ using document root <none> and password file <none>
I20260924 00:18:23.320524 23105 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:23.320569 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:23.320698 23105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:23.321777 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "1082a682311c49198a9ab812ea4f720e"
format_stamp: "Formatted at 2026-09-24 00:18:23 on dist-test-slave-cm7b"
server_key: "8c5f520575b0084da879071765aed10b"
server_key_iv: "c3b80e4bec46e0804bc97d204a97b597"
server_key_version: "encryptionkey@0"
I20260924 00:18:23.322135 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "1082a682311c49198a9ab812ea4f720e"
format_stamp: "Formatted at 2026-09-24 00:18:23 on dist-test-slave-cm7b"
server_key: "8c5f520575b0084da879071765aed10b"
server_key_iv: "c3b80e4bec46e0804bc97d204a97b597"
server_key_version: "encryptionkey@0"
I20260924 00:18:23.323519 23105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:18:23.324240 23120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:23.324410 23105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:23.324499 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "1082a682311c49198a9ab812ea4f720e"
format_stamp: "Formatted at 2026-09-24 00:18:23 on dist-test-slave-cm7b"
server_key: "8c5f520575b0084da879071765aed10b"
server_key_iv: "c3b80e4bec46e0804bc97d204a97b597"
server_key_version: "encryptionkey@0"
I20260924 00:18:23.324571 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:23.336908 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:23.338757 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:23.338910 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:23.342820 23105 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:38667
I20260924 00:18:23.342867 23172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:38667 every 8 connection(s)
I20260924 00:18:23.343277 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:23.344064 23173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:23.346186 23173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e: Bootstrap starting.
I20260924 00:18:23.346694 23173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:23.346925 23173 log.cc:824] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:23.347597 23173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e: No bootstrap required, opened a new log
I20260924 00:18:23.349079 23173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1082a682311c49198a9ab812ea4f720e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 38667 } }
I20260924 00:18:23.349208 23173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:23.349231 23173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1082a682311c49198a9ab812ea4f720e, State: Initialized, Role: FOLLOWER
I20260924 00:18:23.349349 23173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1082a682311c49198a9ab812ea4f720e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 38667 } }
I20260924 00:18:23.349429 23173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:23.349464 23173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:23.349531 23173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:23.350445 23173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1082a682311c49198a9ab812ea4f720e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 38667 } }
I20260924 00:18:23.350570 23173 leader_election.cc:302] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1082a682311c49198a9ab812ea4f720e; no voters:
I20260924 00:18:23.350737 23173 leader_election.cc:288] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:23.350764 23176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:23.350952 23176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [term 1 LEADER]: Becoming Leader. State: Replica: 1082a682311c49198a9ab812ea4f720e, State: Running, Role: LEADER
I20260924 00:18:23.351063 23173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:23.351035 23176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1082a682311c49198a9ab812ea4f720e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 38667 } }
I20260924 00:18:23.351166 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23105
I20260924 00:18:23.351306 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:23.351432 23178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1082a682311c49198a9ab812ea4f720e. Latest consensus state: current_term: 1 leader_uuid: "1082a682311c49198a9ab812ea4f720e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1082a682311c49198a9ab812ea4f720e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 38667 } } }
I20260924 00:18:23.351482 1202 external_mini_cluster.cc:1428] Setting key a675782f5f9a226782532d3d4f84fb21
I20260924 00:18:23.351501 23178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:23.351801 23177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1082a682311c49198a9ab812ea4f720e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1082a682311c49198a9ab812ea4f720e" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 38667 } } }
I20260924 00:18:23.351934 23177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e [sys.catalog]: This master's current role is: LEADER
W20260924 00:18:23.352905 23194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:23.352970 23194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:23.353036 23181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:23.353226 23181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:23.354784 23181 catalog_manager.cc:1347] Generated new cluster ID: 4d044ada70cf4d18a18f618ade58f898
I20260924 00:18:23.354856 23181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 00:18:23 dist-test-slave-cm7b krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:23.361696 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:23.351910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60589 (local address 127.1.44.190:38667)
0924 00:18:23.352071 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:23.352079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:23.352356 (+ 277us) server_negotiation.cc:407] Connection header received
0924 00:18:23.352636 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:23.352644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:23.352707 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:23.352859 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:23.353741 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:23.354820 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:23.355663 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:23.356077 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:23.356206 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:23.358461 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:23.358465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:23.358465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:23.358477 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:23.360202 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:23.360582 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:23.360585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:23.360585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:23.360684 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:23.360903 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:23.360904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:23.360904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:23.361130 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:23.361319 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:23.361385 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 00:18:23.361536 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260924 00:18:23.362921 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:23.370767 23181 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:23.371459 23181 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:23.373363 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:23.364288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50689 (local address 127.1.44.190:38667)
0924 00:18:23.364327 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:23.364329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:23.367366 (+ 3037us) server_negotiation.cc:407] Connection header received
0924 00:18:23.367437 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:23.367438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:23.367462 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:23.367526 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:23.368029 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:23.368496 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:23.369197 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:23.369412 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:23.369493 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:23.370771 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:23.370772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:23.370772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:23.370779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:23.372206 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:23.372561 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:23.372562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:23.372562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:23.372619 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:23.372815 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:23.372816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:23.372816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:23.372925 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:23.373023 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:23.373152 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 00:18:23.373236 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:23.383714 23181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1082a682311c49198a9ab812ea4f720e: Generated new TSK 0
I20260924 00:18:23.383896 23181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:23.389778 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23105
2026-09-24T00:18:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:23 dist-test-slave-cm7b krb5kdc[23204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:23 dist-test-slave-cm7b krb5kdc[23204](info): set up 2 sockets
Sep 24 00:18:23 dist-test-slave-cm7b krb5kdc[23204](info): commencing operation
krb5kdc: starting...
W20260924 00:18:24.411510 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:24 dist-test-slave-cm7b krb5kdc[23204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:24.525529 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46119
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:45851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:46119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:24.603863 23217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:24.604080 23217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:24.604118 23217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:24.605257 23217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:24.605320 23217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:24.605347 23217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:24.605381 23217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:24.605405 23217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:24.606891 23217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:45851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:46119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:46119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:24.607152 23217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:24.607385 23217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:24.612136 23223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:24.612108 23225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:24.612120 23222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:24.612411 23217 server_base.cc:1034] running on GCE node
I20260924 00:18:24.612685 23217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:24.612965 23217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:24.614128 23217 hybrid_clock.cc:648] HybridClock initialized: now 1790209104614112 us; error 45 us; skew 500 ppm
Sep 24 00:18:24 dist-test-slave-cm7b krb5kdc[23204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209104, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:24.616583 23217 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:24.617012 23217 webserver.cc:460] Webserver started at http://127.1.44.190:45139/ using document root <none> and password file <none>
I20260924 00:18:24.617189 23217 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:24.617228 23217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:24.617314 23217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:24.618264 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "52114f849c87451a8f96ef57dd74b664"
format_stamp: "Formatted at 2026-09-24 00:18:24 on dist-test-slave-cm7b"
server_key: "5df8d11dbdd5f3e5782f799ab8b716c3"
server_key_iv: "e933295bf5ac263378d2c296e6eac7fa"
server_key_version: "encryptionkey@0"
I20260924 00:18:24.618564 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "52114f849c87451a8f96ef57dd74b664"
format_stamp: "Formatted at 2026-09-24 00:18:24 on dist-test-slave-cm7b"
server_key: "5df8d11dbdd5f3e5782f799ab8b716c3"
server_key_iv: "e933295bf5ac263378d2c296e6eac7fa"
server_key_version: "encryptionkey@0"
I20260924 00:18:24.619863 23217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:18:24.620568 23232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:24.620780 23217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:24.620854 23217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "52114f849c87451a8f96ef57dd74b664"
format_stamp: "Formatted at 2026-09-24 00:18:24 on dist-test-slave-cm7b"
server_key: "5df8d11dbdd5f3e5782f799ab8b716c3"
server_key_iv: "e933295bf5ac263378d2c296e6eac7fa"
server_key_version: "encryptionkey@0"
I20260924 00:18:24.620945 23217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:24.628960 23217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:24.630795 23217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:24.630944 23217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:24.634889 23217 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:46119
I20260924 00:18:24.634918 23284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:46119 every 8 connection(s)
I20260924 00:18:24.635339 23217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:24.636147 23285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:24.638216 23285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664: Bootstrap starting.
I20260924 00:18:24.638708 23285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:24.638945 23285 log.cc:824] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:24.639634 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23217
I20260924 00:18:24.639679 23285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664: No bootstrap required, opened a new log
I20260924 00:18:24.639758 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:24.639938 1202 external_mini_cluster.cc:1428] Setting key 77d2fb3797ffd9cf520553b0929d3ce9
I20260924 00:18:24.642267 23285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52114f849c87451a8f96ef57dd74b664" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46119 } }
I20260924 00:18:24.642454 23285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:24.642490 23285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52114f849c87451a8f96ef57dd74b664, State: Initialized, Role: FOLLOWER
I20260924 00:18:24.642609 23285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52114f849c87451a8f96ef57dd74b664" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46119 } }
I20260924 00:18:24.642683 23285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:24.642712 23285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:24.642758 23285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:24.643446 23285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52114f849c87451a8f96ef57dd74b664" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46119 } }
I20260924 00:18:24.643569 23285 leader_election.cc:302] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52114f849c87451a8f96ef57dd74b664; no voters:
I20260924 00:18:24.643759 23285 leader_election.cc:288] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:24.643778 23290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:24.643888 23290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [term 1 LEADER]: Becoming Leader. State: Replica: 52114f849c87451a8f96ef57dd74b664, State: Running, Role: LEADER
I20260924 00:18:24.643970 23290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52114f849c87451a8f96ef57dd74b664" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46119 } }
I20260924 00:18:24.644016 23285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:24.644263 23292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52114f849c87451a8f96ef57dd74b664. Latest consensus state: current_term: 1 leader_uuid: "52114f849c87451a8f96ef57dd74b664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52114f849c87451a8f96ef57dd74b664" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46119 } } }
I20260924 00:18:24.644488 23292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:24.644425 23291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52114f849c87451a8f96ef57dd74b664" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52114f849c87451a8f96ef57dd74b664" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 46119 } } }
I20260924 00:18:24.644749 23291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:24.644845 23297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:24.645361 23297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:24.645639 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:24.640402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51849 (local address 127.1.44.190:46119)
0924 00:18:24.640585 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:24.640593 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:24.640617 (+ 24us) server_negotiation.cc:407] Connection header received
0924 00:18:24.641029 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:24.641036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:24.641078 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:24.641185 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:24.641407 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:24.642251 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:24.643400 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:24.643942 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:24.644339 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:24.644746 (+ 407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:24.644750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:24.644751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:24.644953 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:24.644989 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:24.645003 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:24.645090 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:24.645353 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 00:18:24.645416 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260924 00:18:24.646642 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:24.647475 23297 catalog_manager.cc:1347] Generated new cluster ID: 569144dcbb1a49028578831668ec987b
I20260924 00:18:24.647527 23297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:24.654457 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:24.651291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38261 (local address 127.1.44.190:46119)
0924 00:18:24.651331 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:24.651334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:24.651342 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:18:24.651361 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:24.651362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:24.651372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:24.651429 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:24.651576 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:24.652286 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:24.653232 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:24.653546 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:24.653621 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:24.654231 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:24.654233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:24.654233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:24.654237 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:24.654253 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:24.654258 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:24.654315 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:24.654375 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 00:18:24.654393 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:24.667042 23297 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:24.667815 23297 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:24.683050 23297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52114f849c87451a8f96ef57dd74b664: Generated new TSK 0
I20260924 00:18:24.683248 23297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:24.691715 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23217
2026-09-24T00:18:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:24 dist-test-slave-cm7b krb5kdc[23316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:24 dist-test-slave-cm7b krb5kdc[23316](info): set up 2 sockets
Sep 24 00:18:24 dist-test-slave-cm7b krb5kdc[23316](info): commencing operation
krb5kdc: starting...
W20260924 00:18:25.735450 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:25 dist-test-slave-cm7b krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:25.857656 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:45221
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:41527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:45221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:25.936518 23329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:25.936718 23329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:25.936741 23329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:25.937885 23329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:25.937927 23329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:25.937942 23329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:25.937980 23329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:25.938010 23329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:25.939491 23329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:41527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:45221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:25.939734 23329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:25.939952 23329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:25.944402 23335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:25.944429 23334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:25.944495 23337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:25.945124 23329 server_base.cc:1034] running on GCE node
I20260924 00:18:25.945350 23329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:25.945629 23329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:25.946794 23329 hybrid_clock.cc:648] HybridClock initialized: now 1790209105946789 us; error 36 us; skew 500 ppm
Sep 24 00:18:25 dist-test-slave-cm7b krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209105, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:25.949258 23329 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:25.949738 23329 webserver.cc:460] Webserver started at http://127.1.44.190:38339/ using document root <none> and password file <none>
I20260924 00:18:25.949961 23329 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:25.950029 23329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:25.950179 23329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:25.951264 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "099106a4fa15426cb512bc74d2207369"
format_stamp: "Formatted at 2026-09-24 00:18:25 on dist-test-slave-cm7b"
server_key: "11bff8af6b70e16ec1759438e4f3a0e4"
server_key_iv: "01d8c66c86d879a0b10cfca5ca39000c"
server_key_version: "encryptionkey@0"
I20260924 00:18:25.951668 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "099106a4fa15426cb512bc74d2207369"
format_stamp: "Formatted at 2026-09-24 00:18:25 on dist-test-slave-cm7b"
server_key: "11bff8af6b70e16ec1759438e4f3a0e4"
server_key_iv: "01d8c66c86d879a0b10cfca5ca39000c"
server_key_version: "encryptionkey@0"
I20260924 00:18:25.952996 23329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:25.954094 23344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:25.954267 23329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 00:18:25.954372 23329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "099106a4fa15426cb512bc74d2207369"
format_stamp: "Formatted at 2026-09-24 00:18:25 on dist-test-slave-cm7b"
server_key: "11bff8af6b70e16ec1759438e4f3a0e4"
server_key_iv: "01d8c66c86d879a0b10cfca5ca39000c"
server_key_version: "encryptionkey@0"
I20260924 00:18:25.954466 23329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:25.957768 23329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:25.959757 23329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:25.959905 23329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:25.963675 23329 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:45221
I20260924 00:18:25.963707 23396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:45221 every 8 connection(s)
I20260924 00:18:25.964022 23329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:25.964962 23397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:25.967046 23397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369: Bootstrap starting.
I20260924 00:18:25.967638 23397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:25.967911 23397 log.cc:824] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:25.968580 23397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369: No bootstrap required, opened a new log
I20260924 00:18:25.970052 23397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099106a4fa15426cb512bc74d2207369" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45221 } }
I20260924 00:18:25.970176 23397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 00:18:25.970223 23397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 099106a4fa15426cb512bc74d2207369, State: Initialized, Role: FOLLOWER
I20260924 00:18:25.970402 23397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099106a4fa15426cb512bc74d2207369" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45221 } }
I20260924 00:18:25.970484 23397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 00:18:25.970526 23397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 00:18:25.970572 23397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:25.971431 23397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099106a4fa15426cb512bc74d2207369" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45221 } }
I20260924 00:18:25.971544 23397 leader_election.cc:302] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 099106a4fa15426cb512bc74d2207369; no voters:
I20260924 00:18:25.971719 23397 leader_election.cc:288] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 00:18:25.971777 23400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:25.971931 23400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [term 1 LEADER]: Becoming Leader. State: Replica: 099106a4fa15426cb512bc74d2207369, State: Running, Role: LEADER
I20260924 00:18:25.971948 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23329
I20260924 00:18:25.971967 23397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:25.972054 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:25.972033 23400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099106a4fa15426cb512bc74d2207369" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45221 } }
I20260924 00:18:25.972258 1202 external_mini_cluster.cc:1428] Setting key 3b95d285415acb44eb5fbe12ced98ace
I20260924 00:18:25.972359 23400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 099106a4fa15426cb512bc74d2207369. Latest consensus state: current_term: 1 leader_uuid: "099106a4fa15426cb512bc74d2207369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099106a4fa15426cb512bc74d2207369" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45221 } } }
I20260924 00:18:25.972471 23400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:25.972373 23401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "099106a4fa15426cb512bc74d2207369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "099106a4fa15426cb512bc74d2207369" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 45221 } } }
I20260924 00:18:25.972615 23401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369 [sys.catalog]: This master's current role is: LEADER
W20260924 00:18:25.973734 23417 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:25.973795 23417 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:25.973845 23407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:25.973994 23407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:25.975744 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:25.972909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50137 (local address 127.1.44.190:45221)
0924 00:18:25.973742 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:25.973751 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:25.973782 (+ 31us) server_negotiation.cc:407] Connection header received
0924 00:18:25.974437 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:25.974443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:25.974475 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:25.974627 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:25.975309 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:25.975313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:25.975313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:25.975317 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:25.975358 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:25.975373 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:25.975426 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:25.975511 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 00:18:25.975551 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":86,"threads_started":1}
I20260924 00:18:25.976162 23407 catalog_manager.cc:1347] Generated new cluster ID: 4c5d3a4eafdc43728e3e6306ae136755
I20260924 00:18:25.976212 23407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:25.979652 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:25.982084 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:25.981046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44177 (local address 127.1.44.190:45221)
0924 00:18:25.981080 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:25.981082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:25.981207 (+ 125us) server_negotiation.cc:407] Connection header received
0924 00:18:25.981237 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:25.981238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:25.981243 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:25.981311 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:25.981872 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:25.981873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:25.981873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 00:18:25.981877 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:25.981891 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 00:18:25.981896 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:25.981955 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:25.982013 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 00:18:25.982025 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:25.983245 23407 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:25.983366 23407 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:25.987722 23407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 099106a4fa15426cb512bc74d2207369: Generated new TSK 0
I20260924 00:18:25.987877 23407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:25.991263 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23329
2026-09-24T00:18:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1301 ms)
[----------] 18 tests from AuthTokenIssuingTest (23365 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:26 dist-test-slave-cm7b krb5kdc[23428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:26 dist-test-slave-cm7b krb5kdc[23428](info): set up 2 sockets
Sep 24 00:18:26 dist-test-slave-cm7b krb5kdc[23428](info): commencing operation
krb5kdc: starting...
W20260924 00:18:27.023447 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:27.137688 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37323
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37323,127.1.44.189:35097,127.1.44.188:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:27.217119 23441 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 00:18:27.217322 23441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:27.217340 23441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:27.217360 23441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:27.218418 23441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:27.218456 23441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:27.218470 23441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:27.218482 23441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:27.218494 23441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:27.219964 23441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37323,127.1.44.189:35097,127.1.44.188:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:37323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:27.220145 23441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:27.220309 23441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:27.224876 23447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:27.225131 23446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:27.225152 23449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:27.225199 23441 server_base.cc:1034] running on GCE node
I20260924 00:18:27.225462 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:27.225785 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:27.226948 23441 hybrid_clock.cc:648] HybridClock initialized: now 1790209107226941 us; error 41 us; skew 500 ppm
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:27.229405 23441 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:27.229761 23441 webserver.cc:460] Webserver started at http://127.1.44.190:34939/ using document root <none> and password file <none>
I20260924 00:18:27.229943 23441 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:27.229990 23441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:27.230074 23441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:27.231051 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "2efd32f3bb4a467a8369c6d377d3b766"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "d51be49739c6f08f5791d0625dcf6c83"
server_key_iv: "a23de90be2b5837aa9dc287c3ad6e4d0"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.231426 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "2efd32f3bb4a467a8369c6d377d3b766"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "d51be49739c6f08f5791d0625dcf6c83"
server_key_iv: "a23de90be2b5837aa9dc287c3ad6e4d0"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.232714 23441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:18:27.233390 23456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:27.233583 23441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:27.233666 23441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "2efd32f3bb4a467a8369c6d377d3b766"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "d51be49739c6f08f5791d0625dcf6c83"
server_key_iv: "a23de90be2b5837aa9dc287c3ad6e4d0"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.233721 23441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:27.270993 23441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:27.273509 23441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:27.273631 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:27.277520 23441 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:37323
I20260924 00:18:27.277582 23508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:37323 every 8 connection(s)
I20260924 00:18:27.277921 23441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:27.278946 23509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:27.280591 23509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.282553 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23441
I20260924 00:18:27.282675 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:27.282858 1202 external_mini_cluster.cc:1428] Setting key ff31cebd13ecdaa57dbbfa4877e546a9
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.189@KRBTEST.COM; defaulting to no policy
I20260924 00:18:27.291436 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.281259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37323 (local address 127.0.0.1:53816)
0924 00:18:27.281533 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.281561 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.281994 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.282262 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.282265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.282660 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.282910 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.282917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.284090 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.284092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.284932 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.284935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.285666 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.285667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.285721 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.286256 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.286265 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.287736 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.289823 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.289829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.289831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.290032 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.290311 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.290312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.290312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.290389 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.290853 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.290858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.290913 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.290989 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.291162 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":99,"threads_started":1}
I20260924 00:18:27.291440 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.281319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.1.44.190:37323)
0924 00:18:27.281474 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.281482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.281644 (+ 162us) server_negotiation.cc:407] Connection header received
0924 00:18:27.282128 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.282129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.282146 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.282231 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.283039 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.284001 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.285022 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.285549 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.285664 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.287826 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.287835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.287835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.287844 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.289727 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.290119 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.290122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.290123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.290235 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.290461 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.290462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.290463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.290781 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.290872 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.291067 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.291263 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260924 00:18:27.292258 23509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.292625 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.292502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37310)
0924 00:18:27.292552 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.292604 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 00:18:27.292764 23457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
I20260924 00:18:27.293205 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.293115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37316)
0924 00:18:27.293160 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.293188 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 00:18:27.293335 23509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:35097: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab.
Entry for principal kudu/127.1.44.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab.
I20260924 00:18:27.308050 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:35097
--webserver_interface=127.1.44.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab
--principal=kudu/127.1.44.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37323,127.1.44.189:35097,127.1.44.188:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:18:27.342556 23509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } attempt: 1
I20260924 00:18:27.343036 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.342895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37332)
0924 00:18:27.342964 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.343004 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260924 00:18:27.343420 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.343321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37342)
0924 00:18:27.343367 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.343392 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 00:18:27.343513 23509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:35097: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
W20260924 00:18:27.388917 23516 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 00:18:27.389179 23516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:27.389210 23516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:27.389256 23516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:27.390413 23516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:27.390482 23516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:27.390506 23516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:27.390542 23516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:27.390573 23516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:27.392086 23516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37323,127.1.44.189:35097,127.1.44.188:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab
--principal=kudu/127.1.44.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:35097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.44.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:27.392375 23516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:27.392623 23516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 00:18:27.397787 23516 server_base.cc:1034] running on GCE node
W20260924 00:18:27.397748 23524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:27.397770 23521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:27.397748 23522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:27.398240 23516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:27.398557 23516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:27.399719 23516 hybrid_clock.cc:648] HybridClock initialized: now 1790209107399716 us; error 37 us; skew 500 ppm
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:27.402240 23516 init.cc:377] Logged in from keytab as kudu/127.1.44.189@KRBTEST.COM (short username kudu)
I20260924 00:18:27.402736 23516 webserver.cc:460] Webserver started at http://127.1.44.189:41081/ using document root <none> and password file <none>
I20260924 00:18:27.402980 23516 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:27.403054 23516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:27.403251 23516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:27.404332 23516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data/instance:
uuid: "8c57e5b7c33d4f3cb1751ab308f53f25"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "17f7e8076488e6ee39bcc56e3a42d0a7"
server_key_iv: "7a82bb8697de5455868eeffc19aae3a2"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.404714 23516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal/instance:
uuid: "8c57e5b7c33d4f3cb1751ab308f53f25"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "17f7e8076488e6ee39bcc56e3a42d0a7"
server_key_iv: "7a82bb8697de5455868eeffc19aae3a2"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.406085 23516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 00:18:27.406840 23531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:27.407049 23516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 00:18:27.407155 23516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal
uuid: "8c57e5b7c33d4f3cb1751ab308f53f25"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "17f7e8076488e6ee39bcc56e3a42d0a7"
server_key_iv: "7a82bb8697de5455868eeffc19aae3a2"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.407259 23516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:27.411645 23509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } attempt: 2
I20260924 00:18:27.412068 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.411944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37354)
0924 00:18:27.412002 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.412041 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 00:18:27.412438 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.412343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37362)
0924 00:18:27.412388 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.412419 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 00:18:27.412535 23509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:35097: Network error: Client connection negotiation failed: client connection to 127.1.44.189:35097: connect: Connection refused (error 111)
I20260924 00:18:27.470537 23516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:27.472600 23516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:27.472765 23516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:27.476620 23516 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.189:35097
I20260924 00:18:27.476680 23583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.189:35097 every 8 connection(s)
I20260924 00:18:27.477033 23516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
I20260924 00:18:27.477968 23584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:27.479481 23584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.483902 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23516
I20260924 00:18:27.484022 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-1/wal/instance
I20260924 00:18:27.484225 1202 external_mini_cluster.cc:1428] Setting key 3dddc22d4ea2ccc41396ef441068fa8d
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:27.488816 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.480118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.1.44.190:37323)
0924 00:18:27.480196 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.480199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.480386 (+ 187us) server_negotiation.cc:407] Connection header received
0924 00:18:27.480903 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.480905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.480921 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.480971 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.481594 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.482089 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.483089 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.483368 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.483455 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.485789 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.485791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.485791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.485799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.487302 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.487801 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.487802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.487802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.487864 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.488160 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.488160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.488161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.488261 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.488310 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.488685 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.488755 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 00:18:27.489094 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.480104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37323 (local address 127.0.0.1:53830)
0924 00:18:27.480248 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.480297 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.480784 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.481049 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.481053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.481305 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.481495 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.481499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.482185 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.482186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.482993 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.482995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.483476 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.483477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.483604 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.484142 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.484153 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.485695 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.487379 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.487385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.487387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.487719 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.487935 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.487936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.487936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.488088 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.488329 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.488337 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.488505 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.488638 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.488814 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260924 00:18:27.489822 23584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.44.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.188@KRBTEST.COM" created.
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:18:27.499346 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.490168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37376)
0924 00:18:27.490236 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.490247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.490306 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.490862 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.490863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.491026 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.491081 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.491083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.493035 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.493035 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.493550 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.493552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.494071 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.494072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.494127 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.494486 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.494490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.496265 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.498303 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.498305 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.498305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.498467 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.498688 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.498689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.498689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.498738 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.499057 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.499058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.499091 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.499140 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.499274 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260924 00:18:27.499612 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.490225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.1.44.189:35097)
0924 00:18:27.490676 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.490697 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.490722 (+ 25us) server_negotiation.cc:407] Connection header received
0924 00:18:27.490743 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.490745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.490758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.490850 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.491164 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.492914 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.493625 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.493989 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.494051 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.496366 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.496378 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.496379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.496385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.498218 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.498553 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.498556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.498557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.498620 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.498800 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.498801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.498801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.498993 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.499078 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.499175 (+ 97us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.499349 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":415,"thread_start_us":73,"threads_started":1}
I20260924 00:18:27.500202 23584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.500553 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.500433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51296)
0924 00:18:27.500486 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.500527 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 00:18:27.500712 23534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
I20260924 00:18:27.501128 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.501043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51306)
0924 00:18:27.501091 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.501114 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 00:18:27.501250 23584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38269: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
I20260924 00:18:27.503660 23509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } attempt: 3
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab.
Entry for principal kudu/127.1.44.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab.
I20260924 00:18:27.510058 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:38269
--webserver_interface=127.1.44.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab
--principal=kudu/127.1.44.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:38201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:37323,127.1.44.189:35097,127.1.44.188:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:18:27.510332 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.503942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37388)
0924 00:18:27.504006 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.504016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.504063 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.504186 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.504187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.504357 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.504414 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.504416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.504997 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.504998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.505570 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.505572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.505927 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.505928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.506028 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.506460 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.506466 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.507811 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.509141 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.509142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.509142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.509338 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.509599 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.509601 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.509601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.509661 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.509941 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.509943 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.509988 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.510110 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.510220 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260924 00:18:27.510615 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.504002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.1.44.189:35097)
0924 00:18:27.504036 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.504039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.504049 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:18:27.504109 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.504110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.504118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.504161 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.504481 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.504936 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.505652 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.505860 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.505917 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.507898 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.507899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.507900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.507907 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.509041 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.509404 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.509405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.509405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.509481 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.509731 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.509732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.509733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.509858 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.509937 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.510461 (+ 524us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.510541 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:18:27.510979 23509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.511348 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.511246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51312)
0924 00:18:27.511288 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.511315 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 00:18:27.511732 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.511548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51314)
0924 00:18:27.511686 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.511708 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
W20260924 00:18:27.511824 23509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38269: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
I20260924 00:18:27.542979 23509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } attempt: 1
I20260924 00:18:27.543514 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.543367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51316)
0924 00:18:27.543434 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.543479 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 00:18:27.543896 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.543799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51332)
0924 00:18:27.543847 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.543875 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 00:18:27.544034 23509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38269: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
I20260924 00:18:27.550554 23584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } attempt: 1
I20260924 00:18:27.550973 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.550845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51344)
0924 00:18:27.550903 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.550938 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 00:18:27.551296 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.551182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51352)
0924 00:18:27.551248 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.551271 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260924 00:18:27.551390 23584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38269: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
W20260924 00:18:27.590293 23591 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 00:18:27.590557 23591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:27.590595 23591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:27.590639 23591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:27.591821 23591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:27.591890 23591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:27.591917 23591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:27.591950 23591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:27.591981 23591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:27.593504 23591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:38201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:37323,127.1.44.189:35097,127.1.44.188:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab
--principal=kudu/127.1.44.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.44.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:27.593791 23591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:27.594014 23591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:27.598807 23597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:27.598824 23600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:27.598877 23598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:27.599049 23591 server_base.cc:1034] running on GCE node
I20260924 00:18:27.599346 23591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:27.599634 23591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:27.600776 23591 hybrid_clock.cc:648] HybridClock initialized: now 1790209107600760 us; error 33 us; skew 500 ppm
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:27.603142 23591 init.cc:377] Logged in from keytab as kudu/127.1.44.188@KRBTEST.COM (short username kudu)
I20260924 00:18:27.603653 23591 webserver.cc:460] Webserver started at http://127.1.44.188:46801/ using document root <none> and password file <none>
I20260924 00:18:27.603893 23591 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:27.603960 23591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:27.604092 23591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:27.605201 23591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data/instance:
uuid: "bb2eb42e43f54085883068282c80fa00"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "20e4a85f2e400b512f5e30b68a8b42b6"
server_key_iv: "4e973c9557efe0f36b39b2a350545c77"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.605564 23591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal/instance:
uuid: "bb2eb42e43f54085883068282c80fa00"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "20e4a85f2e400b512f5e30b68a8b42b6"
server_key_iv: "4e973c9557efe0f36b39b2a350545c77"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.606829 23591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 00:18:27.607561 23607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:27.607729 23591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:27.607825 23591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal
uuid: "bb2eb42e43f54085883068282c80fa00"
format_stamp: "Formatted at 2026-09-24 00:18:27 on dist-test-slave-cm7b"
server_key: "20e4a85f2e400b512f5e30b68a8b42b6"
server_key_iv: "4e973c9557efe0f36b39b2a350545c77"
server_key_version: "encryptionkey@0"
I20260924 00:18:27.607905 23591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:27.619268 23509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } attempt: 2
I20260924 00:18:27.619519 23584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } attempt: 2
I20260924 00:18:27.619714 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.619570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51366)
0924 00:18:27.619631 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.619670 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 00:18:27.619872 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.619740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51374)
0924 00:18:27.619803 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.619840 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 00:18:27.620044 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.619958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51384)
0924 00:18:27.619996 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.620020 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 00:18:27.620152 23509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38269: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
I20260924 00:18:27.620184 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.620113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51398)
0924 00:18:27.620148 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.620166 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 00:18:27.620275 23584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:38269: Network error: Client connection negotiation failed: client connection to 127.1.44.188:38269: connect: Connection refused (error 111)
I20260924 00:18:27.623250 23591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:27.625075 23591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:27.625183 23591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:27.629026 23591 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.188:38269
I20260924 00:18:27.629076 23659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.188:38269 every 8 connection(s)
I20260924 00:18:27.629442 23591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
I20260924 00:18:27.630424 23660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:27.632031 23660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.634945 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23591
I20260924 00:18:27.635067 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790208846277299-1202-0/minicluster-data/master-2/wal/instance
I20260924 00:18:27.635308 1202 external_mini_cluster.cc:1428] Setting key 0ace8275046a217b05741a9ca0a1689c
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:27.641506 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.632716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.1.44.190:37323)
0924 00:18:27.632774 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.632776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.632981 (+ 205us) server_negotiation.cc:407] Connection header received
0924 00:18:27.633489 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.633491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.633502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.633568 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.634198 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.634669 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.636036 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.636342 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.636415 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.638452 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.638453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.638454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.638461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.639980 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.640456 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.640457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.640458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.640515 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.640758 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.640759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.640759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.640904 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.640974 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.641296 (+ 322us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.641399 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 00:18:27.641618 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.632694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:37323 (local address 127.0.0.1:53844)
0924 00:18:27.632870 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.632903 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.633382 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.633656 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.633662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.633906 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.634092 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.634096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.634760 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.634761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.635949 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.635952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.636444 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.636445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.636556 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.637072 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.637081 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.638391 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.640073 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.640079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.640081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.640383 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.640589 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.640589 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.640590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.640692 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.640962 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.640967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.641120 (+ 153us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.641244 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.641397 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260924 00:18:27.642222 23660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.644495 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.635684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.1.44.188:38269)
0924 00:18:27.635857 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.635862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.635974 (+ 112us) server_negotiation.cc:407] Connection header received
0924 00:18:27.635999 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.636002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.636024 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.636095 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.636781 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.637608 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.638389 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.638805 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.638896 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.641394 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.641402 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.641402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.641409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.643340 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.643724 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.643726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.643727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.643777 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.643975 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.643976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.643976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.644131 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.644269 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.644314 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.644385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
I20260924 00:18:27.649621 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.642549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.1.44.189:35097)
0924 00:18:27.642598 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.642600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.642651 (+ 51us) server_negotiation.cc:407] Connection header received
0924 00:18:27.642719 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.642720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.642731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.642794 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.643117 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.643878 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.644932 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.645245 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.645319 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.647100 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.647101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.647102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.647109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.648530 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.648867 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.648868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.648869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.648920 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.649122 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.649123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.649123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.649237 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.649310 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.649469 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.649536 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 00:18:27.649618 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.642489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:35097 (local address 127.0.0.1:37394)
0924 00:18:27.642603 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.642614 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.642664 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.642810 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.642811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.642988 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.643028 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.643029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.643974 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.643975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.644837 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.644838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.645324 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.645325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.645393 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.645693 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.645700 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.647017 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.648612 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.648613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.648614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.648780 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.648996 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.648997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.648997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.649051 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.649302 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.649303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.649344 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.649436 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.649524 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260924 00:18:27.650005 23660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:27.652423 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.645218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.1.44.190:37323)
0924 00:18:27.645253 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.645254 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.645264 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:18:27.645284 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.645285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.645294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.645352 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.645985 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.646606 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.647384 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.647675 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.647742 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.649962 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.649964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.649964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.649972 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.651324 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.651669 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.651670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.651670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.651717 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.651923 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.651924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.651924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.652032 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.652218 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.652254 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.652326 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:27.656978 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.650256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51422)
0924 00:18:27.650422 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.650436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.650505 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.650809 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.650810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.650919 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.650944 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.650945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.651773 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.651774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.652230 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.652231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.652691 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.652692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.652739 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.653093 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.653098 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.654270 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.656109 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.656109 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.656110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.656264 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.656461 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.656461 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.656462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.656510 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.656769 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.656770 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.656802 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.656848 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.656914 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:18:27.657089 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.650303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.1.44.188:38269)
0924 00:18:27.650348 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.650350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.650497 (+ 147us) server_negotiation.cc:407] Connection header received
0924 00:18:27.650560 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.650561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.650572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.650627 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.651028 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.651705 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.652323 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.652628 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.652684 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.654353 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.654354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.654355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.654361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.656016 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.656334 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.656335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.656336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.656393 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.656578 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.656579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.656580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.656701 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.656766 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.656873 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.656969 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":27}
I20260924 00:18:27.658391 23660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: Bootstrap starting.
I20260924 00:18:27.659379 23660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:27.659673 23660 log.cc:824] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:27.660236 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.653008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37406 (local address 127.1.44.189:35097)
0924 00:18:27.653057 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.653059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.653067 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:18:27.653088 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.653089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.653098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.653144 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.653670 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.654144 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.654739 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.654973 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.655025 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.657518 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.657520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.657520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.657527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.658957 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.659362 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.659363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.659363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.659407 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.659626 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.659627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.659627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.659731 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.660032 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.660072 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.660138 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:27.660640 23660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: No bootstrap required, opened a new log
I20260924 00:18:27.661582 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 00:18:27.662732 23660 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.662858 23660 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2eb42e43f54085883068282c80fa00, State: Initialized, Role: FOLLOWER
I20260924 00:18:27.662940 23660 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.663316 23660 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:27.663297 23666 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.663450 23666 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.662694 23483 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2efd32f3bb4a467a8369c6d377d3b766"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53858
W20260924 00:18:27.665145 23677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:27.665211 23677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:27.665520 23591 master_runner.cc:186] Error getting master registration for 127.1.44.190:37323: OK, instance_id { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" instance_seqno: 1790209107276343 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:18:27.668308 23591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:18:27.711436 23584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } attempt: 3
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:27.719229 23509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } attempt: 3
I20260924 00:18:27.719851 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.711840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51434)
0924 00:18:27.711905 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.711926 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.712002 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.712147 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.712148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.712346 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.712403 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.712404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.713024 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.713026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.714280 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.714284 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.714821 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.714822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.714909 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.715457 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.715462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.716822 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.718633 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.718633 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.718634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.718854 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.719071 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.719072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.719072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.719136 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.719537 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.719538 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.719586 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.719650 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.719748 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 00:18:27.720340 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.711880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.1.44.188:38269)
0924 00:18:27.711926 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.711932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.711980 (+ 48us) server_negotiation.cc:407] Connection header received
0924 00:18:27.712069 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.712070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.712078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.712125 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.712473 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.712936 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.714392 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.714706 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.714780 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.716914 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.716915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.716915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.716931 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.718546 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.718936 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.718938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.718938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.719000 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.719219 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.719220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.719220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.719411 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.719481 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.720172 (+ 691us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.720251 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:18:27.721838 23584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: Bootstrap starting.
I20260924 00:18:27.722569 23584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:27.722940 23584 log.cc:824] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:27.723847 23584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: No bootstrap required, opened a new log
Sep 24 00:18:27 dist-test-slave-cm7b krb5kdc[23428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209107, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:27.725843 23584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.726006 23584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c57e5b7c33d4f3cb1751ab308f53f25, State: Initialized, Role: FOLLOWER
I20260924 00:18:27.726095 23584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.726433 23584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:27.726397 23681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.726554 23681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.727375 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.719494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:38269 (local address 127.0.0.1:51442)
0924 00:18:27.719714 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:27.719727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:27.719772 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:27.720036 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:27.720037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:27.720169 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:27.720209 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.720210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.721130 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.721132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.721963 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:27.721966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.722438 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.722439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:27.722495 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.722893 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:27.722898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:27.724249 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:27.726180 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.726181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.726182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.726395 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.726669 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:27.726670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:27.726670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:27.726734 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:27.727033 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:27.727034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:27.727085 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 00:18:27.727156 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 00:18:27.727275 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196}
I20260924 00:18:27.727384 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.719561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.1.44.188:38269)
0924 00:18:27.719744 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.719746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.719756 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:18:27.719841 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.719842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.719855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.719916 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.720300 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.721039 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.722058 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.722372 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.722423 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.724343 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.724344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.724345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.724352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.726114 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.726483 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.726484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.726485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.726546 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.726806 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.726808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.726808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.726940 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.727005 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.727207 (+ 202us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.727280 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260924 00:18:27.728909 23509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766: Bootstrap starting.
I20260924 00:18:27.729526 23509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:27.729820 23509 log.cc:824] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:27.730526 23509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766: No bootstrap required, opened a new log
I20260924 00:18:27.728526 23516 master_runner.cc:186] Error getting master registration for 127.1.44.190:37323: OK, instance_id { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" instance_seqno: 1790209107276343 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:18:27.731292 23516 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:18:27.731685 23509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.731813 23509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2efd32f3bb4a467a8369c6d377d3b766, State: Initialized, Role: FOLLOWER
I20260924 00:18:27.731930 23509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.732190 23694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.732257 23509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:27.732270 23694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.732388 23483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 00:18:27.732465 23483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:27.733281 23483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
I20260924 00:18:27.733845 23558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2efd32f3bb4a467a8369c6d377d3b766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25"
I20260924 00:18:27.733539 23483 leader_election.cc:288] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [CANDIDATE]: Term 1 election: Requested vote from peers 8c57e5b7c33d4f3cb1751ab308f53f25 (127.1.44.189:35097), bb2eb42e43f54085883068282c80fa00 (127.1.44.188:38269)
I20260924 00:18:27.733969 23558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [term 0 FOLLOWER]: Advancing to term 1
W20260924 00:18:27.734009 23705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:27.734066 23705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:27.734839 23558 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2efd32f3bb4a467a8369c6d377d3b766 in term 1.
I20260924 00:18:27.735069 23634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2efd32f3bb4a467a8369c6d377d3b766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bb2eb42e43f54085883068282c80fa00"
I20260924 00:18:27.735229 23634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:27.736012 23634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2efd32f3bb4a467a8369c6d377d3b766 in term 1.
I20260924 00:18:27.736241 23457 leader_election.cc:302] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2efd32f3bb4a467a8369c6d377d3b766, 8c57e5b7c33d4f3cb1751ab308f53f25; no voters:
I20260924 00:18:27.736361 23694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:27.736482 23694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [term 1 LEADER]: Becoming Leader. State: Replica: 2efd32f3bb4a467a8369c6d377d3b766, State: Running, Role: LEADER
I20260924 00:18:27.736572 23694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } }
W20260924 00:18:27.736713 23692 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:27.736752 23692 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:27.736979 23694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efd32f3bb4a467a8369c6d377d3b766. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.737043 23694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:27.738206 23441 master_runner.cc:263] No leader master found from master 2efd32f3bb4a467a8369c6d377d3b766
I20260924 00:18:27.738248 23441 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:18:27.738786 23714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:27.739027 23714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:27.745751 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.737329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.1.44.190:37323)
0924 00:18:27.737366 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.737368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.738036 (+ 668us) server_negotiation.cc:407] Connection header received
0924 00:18:27.738141 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.738142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.738152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.738182 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.738981 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.739744 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.740843 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.741124 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.741167 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.742779 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.742780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.742780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.742787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.744470 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.744881 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.744882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.744883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.744938 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.745172 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.745173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.745174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.745301 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.745370 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.745591 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.745665 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:27.746500 23634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [term 1 FOLLOWER]: Refusing update from remote peer 2efd32f3bb4a467a8369c6d377d3b766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:27.746516 23558 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [term 1 FOLLOWER]: Refusing update from remote peer 2efd32f3bb4a467a8369c6d377d3b766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:27.746872 23694 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:27.748157 23681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efd32f3bb4a467a8369c6d377d3b766. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.748296 23681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.748499 23711 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:27.749818 23666 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2efd32f3bb4a467a8369c6d377d3b766. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.749913 23666 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.753983 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.737113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.1.44.189:35097)
0924 00:18:27.737151 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.737153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.739907 (+ 2754us) server_negotiation.cc:407] Connection header received
0924 00:18:27.740024 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.740025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.740034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.740063 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.740810 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.741507 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.744009 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.744285 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.744329 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.750702 (+ 6373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.750704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.750704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.750711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.752671 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.753086 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.753087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.753088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.753146 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.753387 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.753388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.753388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.753517 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.753601 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.753801 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.753881 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:18:27.755304 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:27.737680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.1.44.188:38269)
0924 00:18:27.737716 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.737718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.742522 (+ 4804us) server_negotiation.cc:407] Connection header received
0924 00:18:27.743336 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.743337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.743349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.743377 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.748451 (+ 5074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.749187 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.750175 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:27.750503 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:27.750583 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:27.752155 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:27.752157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:27.752157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:27.752164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:27.753790 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.754424 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.754425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.754426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.754483 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:27.754686 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:27.754686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:27.754687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:27.754810 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:27.754908 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:27.755087 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 00:18:27.755172 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:27.756508 23718 mvcc.cc:205] Tried to move back new op lower bound from 7332696505323384832 to 7332696505289691136. Current Snapshot: MvccSnapshot[applied={T|T < 7332696505323384832}]
I20260924 00:18:27.756762 23694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.756880 23694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:27.757002 23694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.757097 23694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766 [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:27.757865 23666 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.757949 23666 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.758010 23681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2efd32f3bb4a467a8369c6d377d3b766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2efd32f3bb4a467a8369c6d377d3b766" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 37323 } } peers { permanent_uuid: "8c57e5b7c33d4f3cb1751ab308f53f25" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 35097 } } peers { permanent_uuid: "bb2eb42e43f54085883068282c80fa00" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 38269 } } }
I20260924 00:18:27.758096 23681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:27.758415 23721 mvcc.cc:205] Tried to move back new op lower bound from 7332696505323384832 to 7332696505289691136. Current Snapshot: MvccSnapshot[applied={T|T < 7332696505323384832 or (T in {7332696505323384832})}]
I20260924 00:18:27.767642 23714 catalog_manager.cc:1347] Generated new cluster ID: 058ca8d518944317853951cba2d0de0c
I20260924 00:18:27.767719 23714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:27.820385 23714 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:27.821094 23714 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:27.835481 23714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2efd32f3bb4a467a8369c6d377d3b766: Generated new TSK 0
I20260924 00:18:27.835685 23714 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 00:18:27.859165 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.858105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.1.44.189:35097)
0924 00:18:27.858152 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.858155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.858499 (+ 344us) server_negotiation.cc:407] Connection header received
0924 00:18:27.858531 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.858532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.858555 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.858617 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.859097 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37422: BlockingRecv error: recv got EOF from 127.0.0.1:37422 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 00:18:27.859171 23512 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.858125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53872 (local address 127.1.44.190:37323)
0924 00:18:27.858189 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.858191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.858457 (+ 266us) server_negotiation.cc:407] Connection header received
0924 00:18:27.858554 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.858555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.858566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.858597 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.859104 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53872: BlockingRecv error: recv got EOF from 127.0.0.1:53872 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 00:18:27.859970 23679 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.858439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.1.44.188:38269)
0924 00:18:27.858479 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.858481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.859401 (+ 920us) server_negotiation.cc:407] Connection header received
0924 00:18:27.859501 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.859502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.859513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.859542 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.859899 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51456: BlockingRecv error: recv got EOF from 127.0.0.1:51456 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 00:18:27.864017 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.861065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.1.44.189:35097)
0924 00:18:27.861110 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.861112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.863408 (+ 2296us) server_negotiation.cc:407] Connection header received
0924 00:18:27.863523 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.863524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.863543 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.863611 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.863968 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37426: BlockingRecv error: recv got EOF from 127.0.0.1:37426 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 00:18:27.866837 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.866348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.1.44.189:35097)
0924 00:18:27.866382 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.866384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.866391 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:18:27.866414 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.866414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.866423 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.866486 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.866791 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37440: BlockingRecv error: recv got EOF from 127.0.0.1:37440 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 00:18:27.870340 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.869757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.1.44.189:35097)
0924 00:18:27.869793 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.869794 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.869865 (+ 71us) server_negotiation.cc:407] Connection header received
0924 00:18:27.869930 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.869931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.869941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.870003 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.870299 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37442: BlockingRecv error: recv got EOF from 127.0.0.1:37442 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 00:18:27.876446 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.875461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.1.44.189:35097)
0924 00:18:27.875498 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.875499 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.875630 (+ 131us) server_negotiation.cc:407] Connection header received
0924 00:18:27.875972 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.875974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.875986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.876057 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.876399 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37446: BlockingRecv error: recv got EOF from 127.0.0.1:37446 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 00:18:27.886262 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.885505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.1.44.189:35097)
0924 00:18:27.885560 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.885562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.885671 (+ 109us) server_negotiation.cc:407] Connection header received
0924 00:18:27.885732 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.885733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.885746 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.885806 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.886202 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37452: BlockingRecv error: recv got EOF from 127.0.0.1:37452 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 00:18:27.904307 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.903501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.1.44.189:35097)
0924 00:18:27.903549 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.903552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.903714 (+ 162us) server_negotiation.cc:407] Connection header received
0924 00:18:27.903761 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.903762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.903775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.903844 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.904249 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37458: BlockingRecv error: recv got EOF from 127.0.0.1:37458 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 00:18:27.938047 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:27.937382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.1.44.189:35097)
0924 00:18:27.937571 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:27.937574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:27.937585 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:18:27.937623 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:27.937624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:27.937635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:27.937699 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:27.937999 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37470: BlockingRecv error: recv got EOF from 127.0.0.1:37470 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260924 00:18:28.004115 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:28.003262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.1.44.189:35097)
0924 00:18:28.003474 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.003476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.003548 (+ 72us) server_negotiation.cc:407] Connection header received
0924 00:18:28.003628 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.003629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.003641 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.003694 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:28.004027 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37476: BlockingRecv error: recv got EOF from 127.0.0.1:37476 (error 108)
Metrics: {"server-negotiator.queue_time_us":182}
W20260924 00:18:28.133993 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:28.133299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.1.44.189:35097)
0924 00:18:28.133407 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.133410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.133430 (+ 20us) server_negotiation.cc:407] Connection header received
0924 00:18:28.133526 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.133527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.133542 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.133610 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:28.133915 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37478: BlockingRecv error: recv got EOF from 127.0.0.1:37478 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260924 00:18:28.391806 23588 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:28.391174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.1.44.189:35097)
0924 00:18:28.391279 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.391282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.391325 (+ 43us) server_negotiation.cc:407] Connection header received
0924 00:18:28.391388 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.391389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.391404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.391456 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:28.391758 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37484: BlockingRecv error: recv got EOF from 127.0.0.1:37484 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260924 00:18:28.665650 23677 catalog_manager.cc:1259] Loaded cluster ID: 058ca8d518944317853951cba2d0de0c
I20260924 00:18:28.665730 23677 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: loading cluster ID for follower catalog manager: success
I20260924 00:18:28.666476 23677 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: acquiring CA information for follower catalog manager: success
I20260924 00:18:28.666738 23677 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb2eb42e43f54085883068282c80fa00: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:18:28.737193 23692 catalog_manager.cc:1259] Loaded cluster ID: 058ca8d518944317853951cba2d0de0c
I20260924 00:18:28.737303 23692 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: loading cluster ID for follower catalog manager: success
I20260924 00:18:28.738171 23692 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: acquiring CA information for follower catalog manager: success
I20260924 00:18:28.738480 23692 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c57e5b7c33d4f3cb1751ab308f53f25: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:18:28.907761 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:28.904952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.1.44.189:35097)
0924 00:18:28.905070 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.905072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.905149 (+ 77us) server_negotiation.cc:407] Connection header received
0924 00:18:28.905252 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.905253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.905336 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.905427 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:28.905590 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.906107 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.906918 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.907127 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.907204 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:28.907407 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:28.907528 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:28.907622 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:28.907642 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 00:18:28.907664 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":51,"threads_started":1}
I20260924 00:18:28.912014 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:28.909523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.1.44.188:38269)
0924 00:18:28.909629 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.909631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.909641 (+ 10us) server_negotiation.cc:407] Connection header received
0924 00:18:28.909689 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.909690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.909724 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.909792 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:28.909895 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.910510 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.911303 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.911535 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.911589 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:28.911714 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:28.911848 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:28.911904 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:28.911934 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 00:18:28.911954 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
I20260924 00:18:28.916402 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:28.913637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.1.44.189:35097)
0924 00:18:28.913685 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.913687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.913694 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:18:28.913772 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.913773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.913784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.913846 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:28.914001 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.914624 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.915654 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.915944 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.916010 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:28.916153 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:28.916184 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:28.916245 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:28.916296 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 00:18:28.916315 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 00:18:28.916685 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:28.913901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.1.44.188:38269)
0924 00:18:28.913939 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.913941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.914083 (+ 142us) server_negotiation.cc:407] Connection header received
0924 00:18:28.914168 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.914168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.914178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.914204 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:28.914404 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.915047 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.915923 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.916202 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.916259 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:28.916421 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:28.916449 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:28.916476 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:28.916577 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 00:18:28.916601 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:28.916883 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:28.913563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.1.44.190:37323)
0924 00:18:28.913674 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:28.913676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:28.913849 (+ 173us) server_negotiation.cc:407] Connection header received
0924 00:18:28.913896 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:28.913897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:28.913960 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:28.914039 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:28.914147 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.914837 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.915815 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:28.916146 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:28.916206 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:28.916451 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:28.916577 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:28.916630 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:28.916779 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 00:18:28.916804 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260924 00:18:28.918437 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23441
I20260924 00:18:28.923686 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23516
I20260924 00:18:28.928328 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23591
2026-09-24T00:18:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2936 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 00:18:28 dist-test-slave-cm7b krb5kdc[23809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 00:18:28 dist-test-slave-cm7b krb5kdc[23809](info): set up 2 sockets
Sep 24 00:18:28 dist-test-slave-cm7b krb5kdc[23809](info): commencing operation
krb5kdc: starting...
W20260924 00:18:29.955546 1202 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/test-user.keytab.
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T00:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T00:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.44.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
Entry for principal kudu/127.1.44.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab.
I20260924 00:18:30.074126 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:32789
--webserver_interface=127.1.44.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:32789,127.1.44.189:45611,127.1.44.188:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 00:18:30.154877 23822 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 00:18:30.155086 23822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:30.155103 23822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:30.155122 23822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:30.156293 23822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:30.156335 23822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:30.156349 23822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:30.156363 23822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:30.156374 23822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:30.157832 23822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:32789,127.1.44.189:45611,127.1.44.188:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.190.keytab
--principal=kudu/127.1.44.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.190:32789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.44.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:30.158092 23822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:30.158324 23822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:30.163239 23830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:30.163244 23828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:30.163290 23827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:30.163398 23822 server_base.cc:1034] running on GCE node
I20260924 00:18:30.163771 23822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:30.164057 23822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:30.165198 23822 hybrid_clock.cc:648] HybridClock initialized: now 1790209110165187 us; error 29 us; skew 500 ppm
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:30.167662 23822 init.cc:377] Logged in from keytab as kudu/127.1.44.190@KRBTEST.COM (short username kudu)
I20260924 00:18:30.168128 23822 webserver.cc:460] Webserver started at http://127.1.44.190:35969/ using document root <none> and password file <none>
I20260924 00:18:30.168351 23822 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:30.168418 23822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:30.168567 23822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:30.169574 23822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data/instance:
uuid: "6e09fdb70230499fa5325045cd9963fe"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "c57d5fb1435d5c6a14ef9da9c39c8f24"
server_key_iv: "8cce3744a1ded501755af8f3a7f493c3"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.169922 23822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal/instance:
uuid: "6e09fdb70230499fa5325045cd9963fe"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "c57d5fb1435d5c6a14ef9da9c39c8f24"
server_key_iv: "8cce3744a1ded501755af8f3a7f493c3"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.171185 23822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:30.171962 23837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:30.172149 23822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 00:18:30.172245 23822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal
uuid: "6e09fdb70230499fa5325045cd9963fe"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "c57d5fb1435d5c6a14ef9da9c39c8f24"
server_key_iv: "8cce3744a1ded501755af8f3a7f493c3"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.172330 23822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:30.189167 23822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:30.191305 23822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:30.191468 23822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:30.195518 23822 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.190:32789
I20260924 00:18:30.195572 23889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.190:32789 every 8 connection(s)
I20260924 00:18:30.195935 23822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/data/info.pb
I20260924 00:18:30.196890 23890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:30.198357 23890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.198829 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23822
I20260924 00:18:30.198953 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-0/wal/instance
I20260924 00:18:30.199155 1202 external_mini_cluster.cc:1428] Setting key ef57759b697776403ec5b783e9b6a50e
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.189@KRBTEST.COM; defaulting to no policy
I20260924 00:18:30.209430 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.199319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56498 (local address 127.1.44.190:32789)
0924 00:18:30.199464 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.199469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.199521 (+ 52us) server_negotiation.cc:407] Connection header received
0924 00:18:30.200174 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.200176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.200192 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.200266 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.200996 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.202088 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.203444 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.203792 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.203915 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.206185 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.206192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.206192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.206206 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.207773 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.208235 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.208238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.208238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.208318 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.208548 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.208550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.208550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.208759 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.208845 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.209050 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.209251 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.189@KRBTEST.COM" created.
I20260924 00:18:30.209429 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.199045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:32789 (local address 127.0.0.1:56498)
0924 00:18:30.199235 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.199287 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.200029 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.200305 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.200308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.200688 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.200878 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.200882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.202182 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.202184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.203365 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.203368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.203916 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.203918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.204005 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.204511 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.204522 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.206084 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.207872 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.207878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.207881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.208156 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.208387 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.208388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.208389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.208477 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.208836 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.208840 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.208903 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.208996 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.209187 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260924 00:18:30.210541 23890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.212103 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.211980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39784)
0924 00:18:30.212036 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.212080 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 00:18:30.212257 23841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
I20260924 00:18:30.212687 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.212606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39800)
0924 00:18:30.212647 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.212670 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 00:18:30.212803 23890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:45611: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab.
Entry for principal kudu/127.1.44.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab.
I20260924 00:18:30.226182 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:45611
--webserver_interface=127.1.44.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab
--principal=kudu/127.1.44.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:32789,127.1.44.189:45611,127.1.44.188:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:18:30.262036 23890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } attempt: 1
I20260924 00:18:30.262598 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.262449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39810)
0924 00:18:30.262527 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.262569 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 00:18:30.262979 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.262902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39816)
0924 00:18:30.262941 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.262964 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 00:18:30.263084 23890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:45611: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
W20260924 00:18:30.307029 23897 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 00:18:30.307256 23897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:30.307274 23897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:30.307340 23897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:30.308518 23897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:30.308562 23897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:30.308576 23897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:30.308589 23897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:30.308643 23897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:30.310108 23897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:32789,127.1.44.189:45611,127.1.44.188:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.189.keytab
--principal=kudu/127.1.44.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.189:45611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.44.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:30.310376 23897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:30.310626 23897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:30.315450 23903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:30.315471 23902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:30.315456 23905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:30.315761 23897 server_base.cc:1034] running on GCE node
I20260924 00:18:30.316022 23897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:30.316344 23897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:30.317512 23897 hybrid_clock.cc:648] HybridClock initialized: now 1790209110317499 us; error 28 us; skew 500 ppm
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:30.319919 23897 init.cc:377] Logged in from keytab as kudu/127.1.44.189@KRBTEST.COM (short username kudu)
I20260924 00:18:30.320405 23897 webserver.cc:460] Webserver started at http://127.1.44.189:35859/ using document root <none> and password file <none>
I20260924 00:18:30.320639 23897 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:30.320706 23897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:30.320839 23897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:30.321863 23897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data/instance:
uuid: "e653bdc8fe78420abf6f01666449090a"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "c319f2c37b99ef59e2f45659bea2b301"
server_key_iv: "a024e1b93fe286e8307874517af3e699"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.322234 23897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal/instance:
uuid: "e653bdc8fe78420abf6f01666449090a"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "c319f2c37b99ef59e2f45659bea2b301"
server_key_iv: "a024e1b93fe286e8307874517af3e699"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.323622 23897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 00:18:30.324321 23912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:30.324492 23897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:30.324580 23897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal
uuid: "e653bdc8fe78420abf6f01666449090a"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "c319f2c37b99ef59e2f45659bea2b301"
server_key_iv: "a024e1b93fe286e8307874517af3e699"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.324661 23897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:30.331228 23890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } attempt: 2
I20260924 00:18:30.331660 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.331538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39832)
0924 00:18:30.331592 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.331629 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 00:18:30.331940 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.331882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39836)
0924 00:18:30.331906 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.331925 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260924 00:18:30.332053 23890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.189:45611: Network error: Client connection negotiation failed: client connection to 127.1.44.189:45611: connect: Connection refused (error 111)
I20260924 00:18:30.341070 23897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:30.342995 23897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:30.343153 23897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:30.346899 23897 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.189:45611
I20260924 00:18:30.346953 23964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.189:45611 every 8 connection(s)
I20260924 00:18:30.347380 23897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/data/info.pb
I20260924 00:18:30.348358 23965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:30.349833 23965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.350874 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23897
I20260924 00:18:30.351004 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-1/wal/instance
I20260924 00:18:30.351257 1202 external_mini_cluster.cc:1428] Setting key e933d8e951b3c573c8de7c739488992b
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.44.188@KRBTEST.COM; defaulting to no policy
I20260924 00:18:30.359076 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.350482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.1.44.190:32789)
0924 00:18:30.350554 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.350557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.350832 (+ 275us) server_negotiation.cc:407] Connection header received
0924 00:18:30.351398 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.351399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.351417 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.351524 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.352128 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.352802 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.353841 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.354042 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.354125 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.356301 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.356303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.356303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.356310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.357671 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.358105 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.358107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.358107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.358190 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.358442 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.358443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.358443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.358555 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.358632 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.358897 (+ 265us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.358979 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 00:18:30.359290 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.350462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:32789 (local address 127.0.0.1:56506)
0924 00:18:30.350637 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.350679 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.351266 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.351600 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.351603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.351840 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.352036 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.352040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.352905 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.352906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.353726 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.353728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.354145 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.354146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.354271 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.354804 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.354812 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.356204 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.357756 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.357776 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.357778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.358009 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.358260 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.358261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.358262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.358371 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.358623 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.358629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.358769 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.358878 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.359041 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260924 00:18:30.359900 23965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.44.188@KRBTEST.COM" created.
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:18:30.368005 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.360153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39846)
0924 00:18:30.360203 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.360216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.360256 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.360601 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.360601 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.360714 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.360753 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.360754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.362033 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.362034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.362883 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.362886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.363395 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.363396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.363498 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.363861 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.363866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.365154 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.366893 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.366894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.366894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.367107 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.367367 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.367367 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.367368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.367426 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.367775 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.367775 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.367811 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.367861 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.367929 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 00:18:30.368121 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.360258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.1.44.189:45611)
0924 00:18:30.360434 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.360437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.360452 (+ 15us) server_negotiation.cc:407] Connection header received
0924 00:18:30.360468 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.360469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.360479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.360559 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.360835 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.361940 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.362963 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.363324 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.363379 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.365254 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.365262 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.365263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.365270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.366805 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.367191 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.367211 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.367211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.367280 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.367494 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.367495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.367496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.367699 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.367798 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.367921 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.368007 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":121,"threads_started":1}
I20260924 00:18:30.368887 23965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.369252 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.369111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38864)
0924 00:18:30.369186 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.369233 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260924 00:18:30.369411 23916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
I20260924 00:18:30.369827 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.369724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38866)
0924 00:18:30.369788 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.369811 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 00:18:30.369987 23965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:35753: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.44.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab.
Entry for principal kudu/127.1.44.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab.
I20260924 00:18:30.377882 1202 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoOH61c/build/release/bin/kudu
/tmp/dist-test-taskoOH61c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:35753
--webserver_interface=127.1.44.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab
--principal=kudu/127.1.44.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.44.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.44.190:32789,127.1.44.189:45611,127.1.44.188:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 00:18:30.419278 23965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } attempt: 1
I20260924 00:18:30.419845 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.419684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38882)
0924 00:18:30.419749 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.419799 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 00:18:30.420320 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.420210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38884)
0924 00:18:30.420262 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.420294 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 00:18:30.420461 23965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:35753: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
I20260924 00:18:30.423256 23890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } attempt: 3
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:18:30.430853 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.423596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39854)
0924 00:18:30.423636 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.423668 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.423721 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.423869 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.423870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.424033 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.424096 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.424097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.424758 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.424759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.425284 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.425285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.425691 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.425692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.425773 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.426241 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.426245 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.428153 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.429804 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.429805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.429805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.429979 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.430195 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.430196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.430197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.430254 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.430518 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.430519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.430565 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.430653 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.430752 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 00:18:30.430853 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.423636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.1.44.189:45611)
0924 00:18:30.423692 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.423694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.423706 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:18:30.423800 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.423801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.423812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.423863 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.424177 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.424675 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.425388 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.425615 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.425694 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.428244 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.428246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.428247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.428254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.429701 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.430065 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.430067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.430067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.430127 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.430333 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.430334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.430334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.430450 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.430526 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.430677 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.430755 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 00:18:30.431470 23890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.431808 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.431708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38898)
0924 00:18:30.431755 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.431783 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 00:18:30.432101 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.432012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38914)
0924 00:18:30.432060 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.432082 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 00:18:30.432189 23890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:35753: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
W20260924 00:18:30.460963 23972 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 00:18:30.461225 23972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 00:18:30.461256 23972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 00:18:30.461298 23972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 00:18:30.462577 23972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 00:18:30.462639 23972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 00:18:30.462664 23972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 00:18:30.462692 23972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 00:18:30.462724 23972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 00:18:30.463289 23890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } attempt: 1
I20260924 00:18:30.463784 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.463642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38928)
0924 00:18:30.463713 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.463755 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260924 00:18:30.464104 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.464010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38944)
0924 00:18:30.464054 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.464082 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 00:18:30.464190 23890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:35753: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
I20260924 00:18:30.464547 23972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.44.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.44.190:32789,127.1.44.189:45611,127.1.44.188:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/krb5kdc/kudu_127.1.44.188.keytab
--principal=kudu/127.1.44.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.44.188:35753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.44.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 23:03:17 UTC on ccycloud.quasar-qwsgns.root.comops.site
build id 4668911
I20260924 00:18:30.464830 23972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 00:18:30.465060 23972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 00:18:30.470059 23980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:30.470116 23977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 00:18:30.470103 23978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 00:18:30.470242 23972 server_base.cc:1034] running on GCE node
I20260924 00:18:30.470525 23972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 00:18:30.470824 23972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 00:18:30.471978 23972 hybrid_clock.cc:648] HybridClock initialized: now 1790209110471956 us; error 38 us; skew 500 ppm
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 00:18:30.474417 23972 init.cc:377] Logged in from keytab as kudu/127.1.44.188@KRBTEST.COM (short username kudu)
I20260924 00:18:30.474867 23972 webserver.cc:460] Webserver started at http://127.1.44.188:41785/ using document root <none> and password file <none>
I20260924 00:18:30.475122 23972 fs_manager.cc:359] Metadata directory not provided
I20260924 00:18:30.475176 23972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 00:18:30.475333 23972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 00:18:30.476327 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data/instance:
uuid: "eddd8268ff5e45978fa9180b4df5716e"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "ac8b1270f35f0a0e865aff131db07cd3"
server_key_iv: "30c1b7089b87551cf7e456ba71095eea"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.476642 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal/instance:
uuid: "eddd8268ff5e45978fa9180b4df5716e"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "ac8b1270f35f0a0e865aff131db07cd3"
server_key_iv: "30c1b7089b87551cf7e456ba71095eea"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.477931 23972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 00:18:30.478621 23987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 00:18:30.478792 23972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 00:18:30.478906 23972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data,/tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal
uuid: "eddd8268ff5e45978fa9180b4df5716e"
format_stamp: "Formatted at 2026-09-24 00:18:30 on dist-test-slave-cm7b"
server_key: "ac8b1270f35f0a0e865aff131db07cd3"
server_key_iv: "30c1b7089b87551cf7e456ba71095eea"
server_key_version: "encryptionkey@0"
I20260924 00:18:30.478981 23972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 00:18:30.488641 23965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } attempt: 2
I20260924 00:18:30.489132 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.488997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38948)
0924 00:18:30.489057 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.489090 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 00:18:30.489517 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.489409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38958)
0924 00:18:30.489463 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.489493 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 00:18:30.489689 23965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.44.188:35753: Network error: Client connection negotiation failed: client connection to 127.1.44.188:35753: connect: Connection refused (error 111)
I20260924 00:18:30.490532 23972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 00:18:30.492610 23972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 00:18:30.492763 23972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 00:18:30.496754 23972 rpc_server.cc:307] RPC server started. Bound to: 127.1.44.188:35753
I20260924 00:18:30.496784 24039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.44.188:35753 every 8 connection(s)
I20260924 00:18:30.497138 23972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/data/info.pb
I20260924 00:18:30.498121 24040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 00:18:30.499754 24040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.502767 1202 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoOH61c/build/release/bin/kudu as pid 23972
I20260924 00:18:30.502894 1202 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoOH61c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790208846277299-1202-0/minicluster-data/master-2/wal/instance
I20260924 00:18:30.503096 1202 external_mini_cluster.cc:1428] Setting key 86a1385ad9752024ac70d539379a56f9
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:30.510139 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.500468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.1.44.190:32789)
0924 00:18:30.500547 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.500549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.500701 (+ 152us) server_negotiation.cc:407] Connection header received
0924 00:18:30.501155 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.501156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.501166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.501225 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.501805 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.502256 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.503452 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.503736 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.503814 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.506658 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.506659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.506660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.506668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.508749 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.509214 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.509215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.509215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.509279 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.509510 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.509511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.509512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.509654 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.509724 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.509965 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.510062 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 00:18:30.510596 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.500408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.190:32789 (local address 127.0.0.1:56516)
0924 00:18:30.500586 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.500620 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.501055 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.501298 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.501301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.501546 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.501732 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.501736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.502327 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.502328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.503327 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.503330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.503834 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.503835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.503999 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.504565 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.504590 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.506540 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.508837 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.508844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.508846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.509124 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.509349 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.509350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.509350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.509444 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.509713 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.509718 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.509827 (+ 109us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.509911 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.510079 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260924 00:18:30.511127 24040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } has no permanent_uuid. Determining permanent_uuid...
I20260924 00:18:30.513098 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.503531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.1.44.188:35753)
0924 00:18:30.503687 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.503698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.503742 (+ 44us) server_negotiation.cc:407] Connection header received
0924 00:18:30.503822 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.503827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.503849 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.503928 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.504613 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.505704 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.506931 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.507372 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.507437 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.510029 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.510035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.510035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.510042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.511846 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.512217 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.512220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.512221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.512285 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.512516 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.512517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.512517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.512680 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.512834 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.512914 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.512990 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:18:30.518003 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.511427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.189:45611 (local address 127.0.0.1:39860)
0924 00:18:30.511476 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.511486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.511526 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.511648 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.511648 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.511881 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.511932 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.511933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.512537 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.512538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.513029 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.513030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.513584 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.513585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.513659 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.514072 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.514077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.515519 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.517032 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.517033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.517034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.517240 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.517455 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.517456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.517456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.517510 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.517790 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.517790 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.517822 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.517871 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.517937 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 00:18:30.518107 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.511511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.1.44.189:45611)
0924 00:18:30.511555 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.511557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.511565 (+ 8us) server_negotiation.cc:407] Connection header received
0924 00:18:30.511589 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.511590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.511599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.511647 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.512008 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.512460 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.513199 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.513473 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.513587 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.515596 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.515597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.515597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.515603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.516963 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.517326 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.517327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.517327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.517387 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.517575 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.517576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.517577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.517724 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.517801 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.517901 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.517983 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:30.518443 24040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.190@KRBTEST.COM
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.188@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:30.522153 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.513824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56532 (local address 127.1.44.190:32789)
0924 00:18:30.513859 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.513860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.514139 (+ 279us) server_negotiation.cc:407] Connection header received
0924 00:18:30.514193 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.514194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.514202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.514245 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.514789 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.515525 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.516702 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.516875 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.516925 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.519523 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.519524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.519525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.519532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.521051 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.521421 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.521422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.521422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.521471 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.521674 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.521675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.521675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.521767 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.521875 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.522019 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.522092 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 00:18:30.524693 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.518711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38976)
0924 00:18:30.518760 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.518771 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.518812 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.518926 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.518927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.519083 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.519116 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.519118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.519752 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.519753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.520208 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.520209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.520589 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.520590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.520637 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.520995 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.520999 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.522224 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.523786 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.523787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.523787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.523931 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.524116 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.524116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.524116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.524155 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.524349 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.524349 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.524378 (+ 29us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.524467 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.524564 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 00:18:30.524693 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.518798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.1.44.188:35753)
0924 00:18:30.518833 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.518835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.518841 (+ 6us) server_negotiation.cc:407] Connection header received
0924 00:18:30.518859 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.518860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.518867 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.518923 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.519171 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.519664 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.520296 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.520513 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.520580 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.522308 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.522310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.522310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.522316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.523712 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.523993 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.523994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.523994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.524057 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.524190 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.524190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.524190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.524299 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.524359 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.524507 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.524606 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":20}
I20260924 00:18:30.526301 24040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: Bootstrap starting.
I20260924 00:18:30.527180 24040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:30.527534 24040 log.cc:824] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: Log is configured to *not* fsync() on all Append() calls
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.44.189@KRBTEST.COM
I20260924 00:18:30.528385 24040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: No bootstrap required, opened a new log
I20260924 00:18:30.530745 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.522979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.1.44.189:45611)
0924 00:18:30.523024 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.523025 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.523074 (+ 49us) server_negotiation.cc:407] Connection header received
0924 00:18:30.523146 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.523147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.523157 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.523251 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.523693 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.524352 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.525424 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.525697 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.525764 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.528176 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.528178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.528178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.528185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.529706 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.530018 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.530019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.530019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.530079 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.530254 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.530255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.530255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.530388 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.530552 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.530592 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.530669 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:30.530629 24040 raft_consensus.cc:348] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.530791 24040 raft_consensus.cc:729] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eddd8268ff5e45978fa9180b4df5716e, State: Initialized, Role: FOLLOWER
I20260924 00:18:30.530874 24040 consensus_queue.cc:260] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.531344 24046 sys_catalog.cc:455] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.531406 24040 sys_catalog.cc:564] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:30.531448 24046 sys_catalog.cc:458] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.532286 1202 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260924 00:18:30.533344 24057 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 00:18:30.533448 24057 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 00:18:30.533588 23864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6e09fdb70230499fa5325045cd9963fe"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56532
I20260924 00:18:30.533553 23972 master_runner.cc:186] Error getting master registration for 127.1.44.190:32789: OK, instance_id { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" instance_seqno: 1790209110194378 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:18:30.535739 23972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:18:30.539306 23890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } attempt: 2
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.190@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:30.547102 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.539568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:38982)
0924 00:18:30.539617 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.539628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.539668 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.539809 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.539810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.539991 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.540052 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.540054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.540922 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.540923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.541790 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.541792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.542239 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.542240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.542302 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.542771 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.542777 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.544362 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.546029 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.546030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.546031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.546206 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.546420 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.546421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.546422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.546479 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.546772 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.546773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.546818 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.546904 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.547005 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 00:18:30.547102 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.539601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38982 (local address 127.1.44.188:35753)
0924 00:18:30.539657 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.539659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.539668 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:18:30.539733 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.539734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.539743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.539806 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.540128 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.540836 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.541876 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.542161 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.542223 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.544452 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.544454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.544454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.544461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.545902 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.546295 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.546296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.546296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.546354 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.546555 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.546556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.546556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.546695 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.546770 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.546927 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.547014 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 00:18:30.548794 23890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe: Bootstrap starting.
I20260924 00:18:30.549536 23890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:30.549842 23890 log.cc:824] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:30.550655 23890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe: No bootstrap required, opened a new log
I20260924 00:18:30.552047 23890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.552192 23890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e09fdb70230499fa5325045cd9963fe, State: Initialized, Role: FOLLOWER
I20260924 00:18:30.552277 23890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.552582 23890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:30.552636 23864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 00:18:30.552759 23864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:30.552585 24060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.552920 24060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.553543 23864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.553785 23864 leader_election.cc:288] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [CANDIDATE]: Term 1 election: Requested vote from peers e653bdc8fe78420abf6f01666449090a (127.1.44.189:45611), eddd8268ff5e45978fa9180b4df5716e (127.1.44.188:35753)
I20260924 00:18:30.554207 24014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e09fdb70230499fa5325045cd9963fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eddd8268ff5e45978fa9180b4df5716e"
I20260924 00:18:30.554345 24014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:30.555363 24014 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e09fdb70230499fa5325045cd9963fe in term 1.
I20260924 00:18:30.555624 23841 leader_election.cc:302] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e09fdb70230499fa5325045cd9963fe, eddd8268ff5e45978fa9180b4df5716e; no voters:
I20260924 00:18:30.555759 24060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 1 FOLLOWER]: Leader election won for term 1
I20260924 00:18:30.556141 24060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [term 1 LEADER]: Becoming Leader. State: Replica: 6e09fdb70230499fa5325045cd9963fe, State: Running, Role: LEADER
I20260924 00:18:30.556228 24060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.556592 24060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e09fdb70230499fa5325045cd9963fe. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.556681 24060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:30.557739 23822 master_runner.cc:186] Error getting master registration for 127.1.44.189:45611: OK, instance_id { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" instance_seqno: 1790209110345913 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 00:18:30.560590 23822 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 00:18:30.560878 24081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 00:18:30.562263 24081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 00:18:30.566704 24014 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [term 1 FOLLOWER]: Refusing update from remote peer 6e09fdb70230499fa5325045cd9963fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:30.567029 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.555424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.1.44.189:45611)
0924 00:18:30.555469 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.555471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.556068 (+ 597us) server_negotiation.cc:407] Connection header received
0924 00:18:30.556163 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.556164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.556173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.556203 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.557008 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.557751 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.559788 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.560505 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.560548 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.562639 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.562641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.562641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.562648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.564625 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.565782 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.565783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.565784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.565855 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.566396 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.566398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.566398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.566533 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.566609 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.566841 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.566919 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:18:30.567090 24060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:30.568614 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.555605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.1.44.188:35753)
0924 00:18:30.555642 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.555644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.558000 (+ 2356us) server_negotiation.cc:407] Connection header received
0924 00:18:30.558092 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.558093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.558104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.558133 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.558818 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.559552 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.560548 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.560823 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.560891 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.562950 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.562951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.562951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.562958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.565394 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.567262 (+ 1868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.567263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.567263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.567321 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.568021 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.568022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.568023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.568154 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.568236 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.568448 (+ 212us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.568527 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:30.568614 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.555973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.1.44.190:32789)
0924 00:18:30.556014 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.556016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.556025 (+ 9us) server_negotiation.cc:407] Connection header received
0924 00:18:30.556049 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.556050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.556060 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.556111 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.557078 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.557768 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.562482 (+ 4714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.562780 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.562824 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.565140 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.565141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.565141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.565148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.566814 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.567775 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.567776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.567777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.567835 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.568041 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.568042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.568042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.568154 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.568236 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.568452 (+ 216us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.568534 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:18:30.568638 24046 sys_catalog.cc:455] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e09fdb70230499fa5325045cd9963fe. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.568719 24046 sys_catalog.cc:458] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.572173 24083 mvcc.cc:205] Tried to move back new op lower bound from 7332696516877099008 to 7332696516838936576. Current Snapshot: MvccSnapshot[applied={T|T < 7332696516877099008}]
I20260924 00:18:30.572263 24079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.572350 24079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:30.572599 24081 catalog_manager.cc:1347] Generated new cluster ID: 9367e51b2331480783a013a14b0fbe76
I20260924 00:18:30.572639 24081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 00:18:30.572739 24046 sys_catalog.cc:455] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.572803 24046 sys_catalog.cc:458] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.578162 24084 mvcc.cc:205] Tried to move back new op lower bound from 7332696516877099008 to 7332696516838936576. Current Snapshot: MvccSnapshot[applied={T|T < 7332696516877099008}]
I20260924 00:18:30.580034 24081 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 00:18:30.580803 24081 catalog_manager.cc:1504] Loading token signing keys...
I20260924 00:18:30.580854 23965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } attempt: 3
Sep 24 00:18:30 dist-test-slave-cm7b krb5kdc[23809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790209110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.44.189@KRBTEST.COM for kudu/127.1.44.188@KRBTEST.COM
I20260924 00:18:30.589767 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.581105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.44.188:35753 (local address 127.0.0.1:39010)
0924 00:18:30.581202 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0924 00:18:30.581214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 00:18:30.581262 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 00:18:30.581459 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 00:18:30.581459 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 00:18:30.581643 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0924 00:18:30.581687 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.581688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.582593 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.582594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.583418 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 00:18:30.583420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.583911 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.583912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 00:18:30.583976 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.584443 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 00:18:30.584448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 00:18:30.585885 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 00:18:30.588540 (+ 2655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.588541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.588541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.588751 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.589021 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 00:18:30.589022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 00:18:30.589022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 00:18:30.589102 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 00:18:30.589433 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 00:18:30.589434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 00:18:30.589483 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 00:18:30.589568 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 00:18:30.589665 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260924 00:18:30.590278 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.581161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39010 (local address 127.1.44.188:35753)
0924 00:18:30.581332 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.581334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.581341 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:18:30.581361 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.581362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.581370 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.581408 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.581764 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.582488 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.583504 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.583793 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.583848 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.585986 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 00:18:30.585987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 00:18:30.585987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 00:18:30.585995 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 00:18:30.588437 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.588854 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.588856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.588856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.588932 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 00:18:30.589188 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 00:18:30.589189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 00:18:30.589189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 00:18:30.589343 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 00:18:30.589442 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.590081 (+ 639us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.590162 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260924 00:18:30.590991 24081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe: Generated new TSK 0
I20260924 00:18:30.591173 24081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 00:18:30.592108 23965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: Bootstrap starting.
I20260924 00:18:30.592818 23965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: Neither blocks nor log segments found. Creating new log.
I20260924 00:18:30.593127 23965 log.cc:824] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: Log is configured to *not* fsync() on all Append() calls
I20260924 00:18:30.604000 23965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: No bootstrap required, opened a new log
I20260924 00:18:30.606623 23965 raft_consensus.cc:348] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.606791 23965 raft_consensus.cc:729] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e653bdc8fe78420abf6f01666449090a, State: Initialized, Role: FOLLOWER
I20260924 00:18:30.606877 23965 consensus_queue.cc:260] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } }
I20260924 00:18:30.607235 23965 sys_catalog.cc:564] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 00:18:30.607527 24091 sys_catalog.cc:455] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.607659 24091 sys_catalog.cc:458] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.608608 23939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e09fdb70230499fa5325045cd9963fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e653bdc8fe78420abf6f01666449090a"
I20260924 00:18:30.608727 23939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [term 0 FOLLOWER]: Advancing to term 1
I20260924 00:18:30.609803 23939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e09fdb70230499fa5325045cd9963fe in term 1.
I20260924 00:18:30.610260 23938 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [term 1 FOLLOWER]: Refusing update from remote peer 6e09fdb70230499fa5325045cd9963fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 00:18:30.610576 24060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 00:18:30.614229 24104 mvcc.cc:205] Tried to move back new op lower bound from 7332696516972371968 to 7332696516838936576. Current Snapshot: MvccSnapshot[applied={T|T < 7332696516877099008}]
I20260924 00:18:30.613075 24091 sys_catalog.cc:455] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e09fdb70230499fa5325045cd9963fe. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.613034 24106 sys_catalog.cc:455] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:30.614857 24091 sys_catalog.cc:458] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.614888 24106 sys_catalog.cc:458] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a [sys.catalog]: This master's current role is: FOLLOWER
I20260924 00:18:30.615856 24103 catalog_manager.cc:1259] Loaded cluster ID: 9367e51b2331480783a013a14b0fbe76
I20260924 00:18:30.615902 24103 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: loading cluster ID for follower catalog manager: success
I20260924 00:18:30.617063 24103 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: acquiring CA information for follower catalog manager: success
I20260924 00:18:30.617372 24103 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e653bdc8fe78420abf6f01666449090a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 00:18:30.617954 23893 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.616813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.1.44.190:32789)
0924 00:18:30.616853 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.616855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.617124 (+ 269us) server_negotiation.cc:407] Connection header received
0924 00:18:30.617297 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.617298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.617311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.617341 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.617884 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56554: BlockingRecv error: recv got EOF from 127.0.0.1:56554 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 00:18:30.618249 23969 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.617171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.1.44.189:45611)
0924 00:18:30.617206 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.617207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.617725 (+ 518us) server_negotiation.cc:407] Connection header received
0924 00:18:30.617819 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.617820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.617829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.617858 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.618197 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39896: BlockingRecv error: recv got EOF from 127.0.0.1:39896 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 00:18:30.618853 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.617460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.1.44.188:35753)
0924 00:18:30.617496 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.617498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.618288 (+ 790us) server_negotiation.cc:407] Connection header received
0924 00:18:30.618483 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.618484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.618493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.618538 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.618796 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39018: BlockingRecv error: recv got EOF from 127.0.0.1:39018 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 00:18:30.622460 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.619872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.1.44.189:45611)
0924 00:18:30.619910 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.619912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.620044 (+ 132us) server_negotiation.cc:407] Connection header received
0924 00:18:30.620122 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.620122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.620150 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.620238 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.620392 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.620879 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.621651 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.621864 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.621950 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.622127 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.622225 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.622302 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.622367 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.622385 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 00:18:30.624811 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.624208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.1.44.188:35753)
0924 00:18:30.624247 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.624249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.624348 (+ 99us) server_negotiation.cc:407] Connection header received
0924 00:18:30.624422 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.624423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.624438 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.624487 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.624750 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39026: BlockingRecv error: recv got EOF from 127.0.0.1:39026 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:30.629251 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.626774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39920 (local address 127.1.44.189:45611)
0924 00:18:30.626805 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.626807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.626814 (+ 7us) server_negotiation.cc:407] Connection header received
0924 00:18:30.626871 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.626871 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.626880 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.626923 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.627016 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.627656 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.628585 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.628786 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.628859 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.629007 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.629037 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.629083 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.629138 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.629154 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 00:18:30.631505 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.630778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.1.44.188:35753)
0924 00:18:30.630817 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.630819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.630976 (+ 157us) server_negotiation.cc:407] Connection header received
0924 00:18:30.631080 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.631081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.631090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.631136 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.631463 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39038: BlockingRecv error: recv got EOF from 127.0.0.1:39038 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:30.642890 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.639578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.1.44.189:45611)
0924 00:18:30.639642 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.639645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.639832 (+ 187us) server_negotiation.cc:407] Connection header received
0924 00:18:30.639946 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.639947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.639958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.639987 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.640177 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.640905 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.642030 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.642310 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.642393 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.642617 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.642651 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.642680 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.642788 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.642810 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260924 00:18:30.646203 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.645390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.1.44.188:35753)
0924 00:18:30.645431 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.645433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.645632 (+ 199us) server_negotiation.cc:407] Connection header received
0924 00:18:30.645728 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.645729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.645741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.645771 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.646146 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39040: BlockingRecv error: recv got EOF from 127.0.0.1:39040 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:30.654961 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.651532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.1.44.189:45611)
0924 00:18:30.651572 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.651574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.651811 (+ 237us) server_negotiation.cc:407] Connection header received
0924 00:18:30.651915 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.651916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.651926 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.651955 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.652134 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.652903 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.654142 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.654431 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.654478 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.654686 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.654720 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.654748 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.654854 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.654876 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 00:18:30.658231 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.657434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39046 (local address 127.1.44.188:35753)
0924 00:18:30.657473 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.657475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.657673 (+ 198us) server_negotiation.cc:407] Connection header received
0924 00:18:30.657770 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.657771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.657783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.657814 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.658173 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39046: BlockingRecv error: recv got EOF from 127.0.0.1:39046 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:30.670879 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.667515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.1.44.189:45611)
0924 00:18:30.667561 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.667563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.667886 (+ 323us) server_negotiation.cc:407] Connection header received
0924 00:18:30.667995 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.667997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.668009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.668038 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.668223 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.668967 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.670107 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.670381 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.670425 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.670616 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.670650 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.670677 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.670778 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.670797 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 00:18:30.674209 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.673422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.1.44.188:35753)
0924 00:18:30.673470 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.673472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.673670 (+ 198us) server_negotiation.cc:407] Connection header received
0924 00:18:30.673767 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.673768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.673779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.673809 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.674151 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39048: BlockingRecv error: recv got EOF from 127.0.0.1:39048 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 00:18:30.695433 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.691657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.1.44.189:45611)
0924 00:18:30.691703 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.691705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.692008 (+ 303us) server_negotiation.cc:407] Connection header received
0924 00:18:30.692124 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.692125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.692136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.692186 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.692664 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.693412 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.694532 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.694820 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.694866 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.695080 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.695116 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.695144 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.695319 (+ 175us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.695351 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 00:18:30.698563 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.697776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.1.44.188:35753)
0924 00:18:30.697815 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.697817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.698034 (+ 217us) server_negotiation.cc:407] Connection header received
0924 00:18:30.698122 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.698123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.698135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.698162 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.698510 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39052: BlockingRecv error: recv got EOF from 127.0.0.1:39052 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:30.734606 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.731822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.1.44.189:45611)
0924 00:18:30.731947 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.731949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.732165 (+ 216us) server_negotiation.cc:407] Connection header received
0924 00:18:30.732298 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.732299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.732310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.732399 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.732565 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.733115 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.733894 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.734104 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.734183 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.734333 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.734376 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.734438 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.734501 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.734519 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260924 00:18:30.744347 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.743606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.1.44.188:35753)
0924 00:18:30.743656 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.743659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.743762 (+ 103us) server_negotiation.cc:407] Connection header received
0924 00:18:30.743863 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.743864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.743875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.743927 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.744267 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39062: BlockingRecv error: recv got EOF from 127.0.0.1:39062 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 00:18:30.811731 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.809440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.1.44.189:45611)
0924 00:18:30.809524 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.809526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.809577 (+ 51us) server_negotiation.cc:407] Connection header received
0924 00:18:30.809637 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.809638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.809648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.809689 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.809809 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.810278 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.811033 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.811279 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.811339 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.811484 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.811512 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.811553 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.811606 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.811622 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260924 00:18:30.814062 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.813521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.1.44.188:35753)
0924 00:18:30.813567 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.813570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.813632 (+ 62us) server_negotiation.cc:407] Connection header received
0924 00:18:30.813690 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.813691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.813703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.813745 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.814000 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39068: BlockingRecv error: recv got EOF from 127.0.0.1:39068 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 00:18:30.945783 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:30.943391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.1.44.189:45611)
0924 00:18:30.943500 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.943502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.943514 (+ 12us) server_negotiation.cc:407] Connection header received
0924 00:18:30.943579 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.943580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.943589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.943642 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:30.943808 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.944284 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.945045 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:30.945221 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:30.945275 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:30.945425 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:30.945451 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:30.945489 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:30.945555 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 00:18:30.945570 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260924 00:18:30.948311 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:30.947707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.1.44.188:35753)
0924 00:18:30.947762 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:30.947765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:30.947791 (+ 26us) server_negotiation.cc:407] Connection header received
0924 00:18:30.947860 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:30.947861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:30.947873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:30.947926 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:30.948244 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39078: BlockingRecv error: recv got EOF from 127.0.0.1:39078 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 00:18:31.205212 24208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e09fdb70230499fa5325045cd9963fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e09fdb70230499fa5325045cd9963fe" member_type: VOTER last_known_addr { host: "127.1.44.190" port: 32789 } } peers { permanent_uuid: "e653bdc8fe78420abf6f01666449090a" member_type: VOTER last_known_addr { host: "127.1.44.189" port: 45611 } } peers { permanent_uuid: "eddd8268ff5e45978fa9180b4df5716e" member_type: VOTER last_known_addr { host: "127.1.44.188" port: 35753 } } }
I20260924 00:18:31.205369 24208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e09fdb70230499fa5325045cd9963fe [sys.catalog]: This master's current role is: LEADER
I20260924 00:18:31.207865 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:31.205455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.1.44.189:45611)
0924 00:18:31.205500 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.205502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.205676 (+ 174us) server_negotiation.cc:407] Connection header received
0924 00:18:31.205739 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.205740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.205751 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.205807 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:31.205942 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.206402 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.207202 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.207381 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.207436 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:31.207600 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:31.207625 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:31.207663 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:31.207774 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 00:18:31.207795 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 00:18:31.210250 24044 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 00:18:31.209621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.1.44.188:35753)
0924 00:18:31.209672 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.209674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.209701 (+ 27us) server_negotiation.cc:407] Connection header received
0924 00:18:31.209758 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.209759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.209767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.209812 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 00:18:31.210178 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39092: BlockingRecv error: recv got EOF from 127.0.0.1:39092 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 00:18:31.533967 24057 catalog_manager.cc:1259] Loaded cluster ID: 9367e51b2331480783a013a14b0fbe76
I20260924 00:18:31.534045 24057 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: loading cluster ID for follower catalog manager: success
I20260924 00:18:31.534966 24057 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: acquiring CA information for follower catalog manager: success
I20260924 00:18:31.535341 24057 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eddd8268ff5e45978fa9180b4df5716e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 00:18:31.725806 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:31.723278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39988 (local address 127.1.44.189:45611)
0924 00:18:31.723407 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.723410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.723490 (+ 80us) server_negotiation.cc:407] Connection header received
0924 00:18:31.723590 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.723590 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.723602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.723681 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:31.723812 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.724318 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.725083 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.725286 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.725357 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:31.725524 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:31.725563 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:31.725623 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:31.725683 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 00:18:31.725706 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260924 00:18:31.729628 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:31.727280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.1.44.188:35753)
0924 00:18:31.727407 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.727409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.727443 (+ 34us) server_negotiation.cc:407] Connection header received
0924 00:18:31.727466 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.727466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.727511 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.727576 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:31.727734 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.728287 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.729006 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.729188 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.729236 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:31.729349 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:31.729463 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:31.729519 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:31.729551 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 00:18:31.729567 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260924 00:18:31.734570 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:31.731652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.1.44.189:45611)
0924 00:18:31.731700 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.731702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.731814 (+ 112us) server_negotiation.cc:407] Connection header received
0924 00:18:31.731866 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.731867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.731877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.731930 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:31.732037 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.732763 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.733900 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.734122 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.734183 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:31.734308 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:31.734342 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:31.734400 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:31.734456 (+ 56us) server_negotiation.cc:299] Negotiation successful
0924 00:18:31.734480 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 00:18:31.734639 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:31.731861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39116 (local address 127.1.44.188:35753)
0924 00:18:31.731895 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.731897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.732112 (+ 215us) server_negotiation.cc:407] Connection header received
0924 00:18:31.732166 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.732167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.732176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.732226 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:31.732324 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.733008 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.733752 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.734046 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.734109 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:31.734383 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:31.734415 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:31.734460 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:31.734548 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 00:18:31.734566 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 00:18:31.734822 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 00:18:31.731507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.1.44.190:32789)
0924 00:18:31.731640 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 00:18:31.731642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 00:18:31.731653 (+ 11us) server_negotiation.cc:407] Connection header received
0924 00:18:31.731710 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 00:18:31.731711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 00:18:31.731760 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 00:18:31.731998 (+ 238us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 00:18:31.732054 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.732825 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.733883 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 00:18:31.734185 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 00:18:31.734244 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 00:18:31.734391 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 00:18:31.734495 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 00:18:31.734704 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0924 00:18:31.734724 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 00:18:31.734749 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260924 00:18:31.741303 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23822
I20260924 00:18:31.746592 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23897
I20260924 00:18:31.752204 1202 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoOH61c/build/release/bin/kudu with pid 23972
2026-09-24T00:18:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2824 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5761 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (265481 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST